Data Analysis Techniques For Engineers

Choosing the right chart is half the battle in data storytelling. This one visual helped me go from “𝐖𝐡𝐢𝐜𝐡 𝐜𝐡𝐚𝐫𝐭 𝐝𝐨 𝐈 𝐮𝐬𝐞?” → “𝐆𝐨𝐭 𝐢𝐭 𝐢𝐧 10 𝐬𝐞𝐜𝐨𝐧𝐝𝐬.”👇 𝐇𝐞𝐫𝐞’𝐬 𝐚 𝐪𝐮𝐢𝐜𝐤 𝐛𝐫𝐞𝐚𝐤𝐝𝐨𝐰𝐧 𝐨𝐟 𝐡𝐨𝐰 𝐭𝐨 𝐜𝐡𝐨𝐨𝐬𝐞 𝐭𝐡𝐞 𝐫𝐢𝐠𝐡𝐭 𝐜𝐡𝐚𝐫𝐭 𝐛𝐚𝐬𝐞𝐝 𝐨𝐧 𝐲𝐨𝐮𝐫 𝐝𝐚𝐭𝐚: 🔹 𝐂𝐨𝐦𝐩𝐚𝐫𝐢𝐬𝐨𝐧? • Few categories → Bar Chart • Over time → Line Chart • Multivariate → Spider Chart • Non-cyclical → Vertical Bar Chart 🔹 𝐑𝐞𝐥𝐚𝐭𝐢𝐨𝐧𝐬𝐡𝐢𝐩? • 2 variables → Scatterplot • 3+ variables → Bubble Chart 🔹 𝐃𝐢𝐬𝐭𝐫𝐢𝐛𝐮𝐭𝐢𝐨𝐧? • Single variable → Histogram • Many points → Line Histogram • 2 variables → Violin Plot 🔹 𝐂𝐨𝐦𝐩𝐨𝐬𝐢𝐭𝐢𝐨𝐧? • Show part of a total → Pie Chart / Tree Map • Over time → Stacked Bar / Area Chart • Add/Subtract → Waterfall Chart 𝐐𝐮𝐢𝐜𝐤 𝐓𝐢𝐩𝐬: • Don’t overload charts; less is more. • Always label axes clearly. • Use color intentionally, not decoratively. • 𝐀𝐬𝐤: What insight should this chart unlock in 5 seconds or less? 𝐑𝐞𝐦𝐞𝐦𝐛𝐞𝐫: • Charts don’t just show data, they tell a story • In storytelling, clarity beats complexity • Don’t aim to impress with fancy visuals, aim to express the insight simply, that’s where the real impact is 💡 ♻️ Save it for later or share it with someone who might find it helpful! 𝐏.𝐒. I share job search tips and insights on data analytics & data science in my free newsletter. Join 14,000+ readers here → https://lnkd.in/dUfe4Ac6

Understanding Material Microstructures through Monte Carlo Simulation of Grain Growth In materials science, the microstructure of a material significantly influences its properties. One critical phenomenon that shapes these microstructures is grain growth—a process where grains within a polycrystalline material coarsen over time, driven by the reduction of surface energy at grain boundaries. What’s fascinating is how a simple set of rules in this simulation can accurately replicate grain boundary dynamics—similar to Conway’s Game of Life, where basic rules governing cell behavior give rise to surprisingly complex patterns. In grain growth simulations, the process is driven by calculating the free energy of each atom in a lattice based on its current crystallographic orientation and comparing it to a random alternative. If the new orientation results in lower or equal energy, it replaces the former, mimicking the natural tendency of materials to minimize surface energy. Despite its simplicity, this approach effectively captures the intricate process of grain coarsening, demonstrating how elegant mathematical models can unravel real-world complexities. One aspect I’m particularly excited about is how this implementation can simulate the evolution of a cold-worked structure during the annealing process. By initializing the simulation with elongated and oriented grains—representative of cold-worked materials—this model reveals how grains gradually recrystallize and coarsen over time, eventually leading to a more equiaxed microstructure. For those curious and intrigued by this simulation, you can try it yourself! Head over to https://lnkd.in/d2Qrd6yK, where I’ve shared the implementation along with detailed instructions. Dive in, experiment, and explore how grain structures evolve right on your own system!

The most expensive engineers I know are always right. They win every technical argument.  They defend every calculation.   They dismiss every suggestion from "non-engineers."  They also deliver projects nobody wants to operate. The uncomfortable truth about engineering: Being technically correct means nothing if your plant operator can't maintain it. I've started doing something different:    → Operator input at 30% design (not 90%)   → Maintenance crew reviews before permitting   → Field techs mark up preliminary layouts   → Junior engineers challenge senior assumptions Why? Because the operator who'll run your design for 20 years knows something you don't. Because that maintenance tech has seen 50 designs fail the same way. Because defending your PE stamp is less important than delivering something that actually works. The hierarchy in engineering is backwards.   We value credentials over experience.   We value calculations over operations.   We value being right over being effective. Your next project has two paths:     → Prove you're the smartest engineer in the room   → Build something that works for the people who'll use it Choose wisely. What "non-engineer" feedback are you currently ignoring that could save your project? #ExperienceEngineer #ExpensiveEngineer #Operator #PE

One of the biggest challenges in data visualization is deciding 𝘸𝘩𝘪𝘤𝘩 chart to use for your data. Here’s a breakdown to guide you through choosing the perfect chart to fit your data’s story: 🟦 𝗖𝗼𝗺𝗽𝗮𝗿𝗶𝘀𝗼𝗻 𝗖𝗵𝗮𝗿𝘁𝘀 If you’re comparing different categories, consider these options: - Embedded Charts – Ideal for comparing across 𝘮𝘢𝘯𝘺 𝘤𝘢𝘵𝘦𝘨𝘰𝘳𝘪𝘦𝘴, giving you a comprehensive view of your data. - Bar Charts – Best for fewer categories where you want a clear, side-by-side comparison. - Spider Charts – Great for showing multivariate data across a few categories; perfect for visualizing strengths and weaknesses in radar-style. 📈 𝗖𝗵𝗮𝗿𝘁𝘀 𝗳𝗼𝗿 𝗗𝗮𝘁𝗮 𝗢𝘃𝗲𝗿 𝗧𝗶𝗺𝗲 When tracking changes or trends over time, pick these charts based on your data structure: - Line Charts – Effective for showing trends across 𝘮𝘢𝘯𝘺 𝘤𝘢𝘵𝘦𝘨𝘰𝘳𝘪𝘦𝘴 over time. Line charts give a sense of continuity. - Vertical Bar Charts – Useful for tracking data over fewer categories, especially when visualizing individual data points within a time frame.    🟩 𝗥𝗲𝗹𝗮𝘁𝗶𝗼𝗻𝘀𝗵𝗶𝗽 𝗖𝗵𝗮𝗿𝘁𝘀 To reveal correlations or relationships between variables: - Scatterplot – Best for displaying the relationship between 𝘵𝘸𝘰 𝘷𝘢𝘳𝘪𝘢𝘣𝘭𝘦𝘴. Perfect for exploring potential patterns and correlations. - Bubble Chart – A go-to choice for three or more variables, giving you an extra dimension for analysis. 🟨 𝗗𝗶𝘀𝘁𝗿𝗶𝗯𝘂𝘁𝗶𝗼𝗻 𝗖𝗵𝗮𝗿𝘁𝘀 Understanding data distribution is essential for statistical analysis. Use these to visualize distribution effectively: - Histogram – Best for a 𝘴𝘪𝘯𝘨𝘭𝘦 𝘷𝘢𝘳𝘪𝘢𝘣𝘭𝘦 with a few data points, ideal for showing the frequency distribution within a dataset. - Line Histogram – Works well when there are many data points to assess distribution over a range. - Scatterplot – Can also illustrate distribution across two variables, especially for seeing clusters or outliers. 🟪 𝗖𝗼𝗺𝗽𝗼𝘀𝗶𝘁𝗶𝗼𝗻 𝗖𝗵𝗮𝗿𝘁𝘀 Show parts of a whole and breakdowns with these: - Tree Map – Ideal for illustrating hierarchical structures or showing the composition of categories as part of a total. - Waterfall Chart – Perfect for showing how individual elements contribute to a cumulative total, with additions and subtractions clearly represented. - Pie Chart – Suitable when you need to show a single share of the total; use sparingly for clarity. - Stacked Bar Chart & Area Chart – Both work well for visualizing composition over time, whether you’re tracking a few or many periods. 💡 Key Takeaways - Comparing across categories? Go for bar charts, embedded charts, or spider charts. - Tracking trends over time? Line or bar charts help capture time-based patterns. - Revealing relationships? Scatter and bubble charts make variable correlations clear. - Exploring distribution? Histograms or scatter plots can showcase data spread. - Showing composition? Use tree maps, waterfall charts, or pie charts for parts of a whole.

Analytics Engineering has levels to it. Level 1: Writing queries Start by mastering the basics: • Strong SQL fundamentals including CTEs, window functions, and subqueries • Querying modern warehouses like Snowflake, BigQuery, Redshift, and Databricks • Understanding table grain, joins, and how nulls behave • Using BI tools such as Looker, Tableau, Power BI, Mode, or Metabase • Light transformations in SQL or Python At this stage, you are focused on answering questions and building confidence in the data. Level 2: Modeling analytics data Move from querying data to shaping it: • Dimensional modeling with facts, dimensions, and star schemas • dbt fundamentals including models, tests, sources, and documentation • Incremental models, snapshots, and slowly changing dimensions • Handling late arriving data and backfills • Basic orchestration with tools like dbt Cloud, Airflow, or Prefect This is where analytics becomes repeatable and reusable across teams. Level 3: Engineering the analytics layer Shift from individual models to systems: • Clear analytics layers from staging to marts • Consistent metric definitions and shared business logic • Version control, code reviews, and CI for analytics projects • Data quality checks using dbt tests or tools like Elementary and Monte Carlo • Environment separation and safe deployments • Reverse ETL with tools like Hightouch or Census Here is where you move from “it works” to “it keeps working.” Level 4: Operating analytics at scale Learn what it takes to support analytics in real companies: • Semantic layers and metric governance with Looker, dbt Semantic Layer, or Cube • Performance tuning with partitioning, clustering, and cost controls • Data observability and incident response using tools like Monte Carlo or Datafold • Data contracts, schema enforcement, and lineage • Access controls, PII handling, and compliance • Tight collaboration with product, finance, and leadership At this point, you are not just building dashboards. You are running the decision making layer of the company. Optional Level 5: Analytics as a platform Not every team needs this, but larger companies do: • Self serve analytics and enablement • Metric APIs and shared data products • Embedded analytics in customer facing products • Company wide analytics standards and education Most teams think they are at Level 3. Most are actually stuck between Level 1 and Level 2. What would you add or change based on your experience?

Blindly Trusting Vendor Data Is a Costly Engineering Mistake Blindly trusting vendor data is one of the most common—and most expensive—mistakes in process engineering. Vendor datasheets are not wrong, but they are not automatically right for your process. As process engineers, we often receive neatly prepared datasheets showing: → Guaranteed performance → High efficiencies → Compliance with standards But here’s the uncomfortable truth 👇 Most equipment failures don’t happen because vendors lied. They happen because engineers stopped questioning. ⚠️ Where Blind Trust Goes Wrong → Rated flow assumed as operating flow → Normal case considered, part-load ignored → Turndown and minimum flow not verified → Fouling, aging, and degradation overlooked → Utilities and site limitations not cross-checked A pump that works perfectly on paper can cavitate in the plant. A heat exchanger that meets duty can fail after six months. A control valve sized “as per datasheet” can generate noise and vibration. 🧠 The Real Engineering Mindset Vendors design equipment. Process engineers design systems. Your responsibility is not to approve numbers. Your responsibility is to protect plant operability and reliability. Always ask: → What is the design basis? → What are the operating and off-design cases? → What happens at minimum flow or maximum turndown? → What will change after two years of operation? ✅ Remember This Vendor data is an input, not a conclusion. Verification is engineering. Blind trust is assumption. If you want to grow as a process engineer, challenge the data—before the plant challenges you. #ProcessEngineering #ProcessDesign #ChemicalEngineering #EPCProjects #PlantDesign #EngineeringReality #ProcessEngineer #MyProcessDesign #ProcessEngineering #ChemicalEngineering #ProcessDesign #Engineering #EngineeringLife #EPC #EPCProjects #PlantDesign #OilAndGas #Refinery #Petrochemical #ProcessEngineer #PlantEngineering #DesignEngineering #EquipmentDesign #EngineeringReality #EngineeringCareer #LearningByDoing #ProfessionalGrowth #EngineeringMindset #MyProcessDesign #EngineeringInsights #ProcessDesignEngineering

Many Digital Twin projects fail. Why? The #1 killer of DT projects is: Data Preprocessing. A true Digital Twin isn't a model. It's an engine. And the fuel for that engine is data. But how do you build the plumbing? How do you get data from your physical asset into your virtual model and then get valuable insights back out? Here’s the 5-step breakdown of the engine you actually need to build: Step 1: Data Acquisition Your engine is useless without fuel. This starts at the source. - IIoT Sensors: These are the nerves of your asset. They measure pressure, temperature, vibration, location—whatever matters. If you can't sense it, you can't twin it 😂 - Real-time Transmission: The data can't be a day old. You need a high-speed data bus (like MQTT, OPC-UA) to transmit that sensor data now. - Data Preprocessing: Again, this is the #1 killer of DT projects. Raw sensor data is dirty. It's noisy, full of gaps, and in the wrong format. You MUST clean, normalize, and filter it before it goes anywhere else. Step 2: The Modeling Now your clean data has somewhere to go. - Digital Twin Construction: You map the data streams to the virtual asset. "Sensor 1A" is now officially the "vibration reading for Pump 7." - Virtual Model: This isn't just a 3D drawing. This is a physics-based or ML model. It understands thermodynamics, material fatigue, or fluid dynamics. This is where the data gets context. Step 3: Analytics This is where the ROI lives. The engine is running. Now, what does it do? Predictive Analytics: Your model takes the data and simulates "what if?" What happens if I increase the load by 20%? When will this specific component fail? - High-Performance Computing (HPC): These complex simulations can't run on a laptop. You need the horsepower to process massive data streams and run complex algorithms instantly. Your data is no longer just describing the past. It's actively predicting and optimizing the future. Step 4 & 5: Security & Standards Your high-performance engine needs a chassis to hold it together. Amateurs forget this. Pros build it first. - Cybersecurity & Privacy: You just connected your most critical physical assets to the cloud. Securing this isn't an afterthought; it's priority #1. - Interoperability Standards: Your sensors, software, and platforms must speak the same language. If you build a proprietary, closed system, you're building technical debt. Plan for an open architecture, always. -------- Follow me for #digitaltwins Links in my profile Florian Huemer

🔬 Thermogravimetric Analysis (TGA) & Differential Scanning Calorimetry (DSC): Understanding Thermal Stability & Phase Behavior of Materials Thermal analysis techniques such as TGA and DSC are indispensable tools in materials science, catalysis, polymers, pharmaceuticals, and energy-related research. These techniques help us understand how materials respond to temperature in terms of mass changes, thermal stability, phase transitions, and reaction energetics. 🔹 What is TGA? Thermogravimetric Analysis (TGA) measures the change in mass of a sample as a function of temperature or time under a controlled atmosphere. 📌 Key Information from TGA: 👉 Moisture and volatile content 👉Thermal stability range 👉Decomposition temperatures 👉Oxidation/reduction behavior 👉Coke or carbon deposition on catalysts 👉Ash or residue content 📌 Common Atmospheres Used: Nitrogen / Argon → inert conditions Air / Oxygen → oxidation studies Hydrogen → reduction behavior 📌 Typical Applications in Catalysis: Determination of coke formation after reaction Stability of fresh vs spent catalysts Decomposition of precursor salts Calcination temperature optimization 🔹 What is DSC? Differential Scanning Calorimetry (DSC) measures the heat flow associated with physical or chemical transitions in a material as a function of temperature. 📌 Information Obtained from DSC: Glass transition temperature (Tg) Melting temperature (Tm) Crystallization temperature (Tc) Phase transitions Reaction enthalpy (endothermic/exothermic events) 📌 Why DSC Matters: Understanding phase purity Identifying polymorphic transformations Studying crystallinity and amorphous content Thermal behavior of polymers and composites 🔹 How to Interpret TGA Curve? A typical TGA curve consists of mass (%) vs temperature: Initial weight loss → moisture or adsorbed species Major weight loss step → decomposition of material Final plateau → residual stable phase 👉 Derivative TGA (DTG) peaks help pinpoint exact decomposition temperatures. 🔹 How to Interpret DSC Curve? DSC plots heat flow vs temperature: Endothermic peaks → melting, evaporation, desorption Exothermic peaks → crystallization, oxidation, curing Peak area → enthalpy change (ΔH) 🔹 Combining TGA + DSC When TGA and DSC are used together: ✅ Correlate mass loss with heat events ✅ Distinguish physical vs chemical transitions ✅ Obtain deeper insight into reaction mechanisms This combined approach is extremely powerful for catalyst development, material design, and process optimization. 💡 Key Takeaway TGA tells how much mass changes, while DSC tells how much energy is involved. Together, they provide a complete picture of a material’s thermal behavior. ✍️ Kanchan Guru DST INSPIRE Fellow (SRF) Department of Chemistry, Manipal University Jaipur Subscribe to Research Decoded newsletter for more insights on characterization & catalysis https://lnkd.in/g74ryQ66

You've heard "garbage in, garbage out" a thousand times. But here's what that actually means: your fancy dashboard is only as good as the data behind it. Quantity is easy to measure—it's just Terabytes. But data quality? Quality is the hard part because it requires discipline, process, and ownership. Data quality and governance are no longer “nice-to-haves.” They define trust across the organization. → Growing demand due to privacy laws like GDPR and CCPA → Core skill required for roles like Data Engineer, Steward, and Architect → Tools like Collibra and Great Expectations now appear in almost every data job description Some numbers speak for themselves: → Data Quality Engineer roles growing 40%+ yearly → Governance Analysts earning around $80K–$120K → Chief Data Officers often crossing $200K+ Clean data isn’t just accuracy—it’s career growth and company credibility. What Good Data Quality Looks Like? Skip the theory. Here's what actually works: → Automated checks that catch issues before they spread → Validation rules that reject bad data at the source → Tracking where data comes from and where it goes → Alerts when something breaks (not after it's been broken for weeks) → Clear ownership so someone actually fixes problems Where in the real world it shows up? 👉This isn't abstract. Here's where data quality makes or breaks things: → Finance: Try explaining bad compliance data to auditors → Healthcare: Patient records need to be right, every time → Retail: Wrong inventory data means lost sales or wasted stock → ML projects: Your model is only as smart as your training data The Real Talk: Data quality feels boring until it's missing. Then suddenly everyone cares. It's not sexy work. Nobody celebrates when pipelines validate correctly. But it's the foundation everything else sits on. Gartner says organizations with formal data governance will see 30% higher ROI by 2026. As data engineers, that’s our call to design solutions that "𝘥𝘰𝘯’𝘵 𝘫𝘶𝘴𝘵 𝘮𝘰𝘷𝘦 𝘥𝘢𝘵𝘢, 𝘣𝘶𝘵 𝘮𝘰𝘷𝘦 𝘵𝘳𝘶𝘴𝘵." Honestly, I feel it's probably more if you count all the fires you don't have to fight. 👉 Folks I admire in this space - George Firican Dylan Anderson Piotr Czarnas 🎯 Mark Freeman II Chad Sanderson Here's a crisp guide on Data Quality & Governance for data engineers! 👇 What's the most annoying, recurring data quality issue you've had to fix lately? I'll go first: dates stored as strings. 🤦♂️