ROC (Receiver Operation Characteristics) – curve is a graph that shows how classifiers performs by plotting the true positive and false positive rates. It is used to evaluate the performance of binary classification models by illustrating the trade-off between True…
Blog Archives
Data science with Microsoft Fabric – Plotting ROC curve and distribution of scores
Tagged with: ai, artificial-intelligence, data science, Distributions, Fabric, machine learning, Microsoft Fabric, Python, R, Spark
Posted in Azure Machine Learning, Fabric, R, Spark
Posted in Azure Machine Learning, Fabric, R, Spark
Data engineering functions on large datasets in Microsoft Fabric
Data engineering and even simple data wrangling functions in Fabric can make several tasks faster, when you know know, which package (language) to choose. By comparing Python Pandas with PySpark Pandas (Koalas), we will see that there are huge performance…
Tagged with: data analysis, data science, Fabric, machine learning, Microsoft Fabric, pandas, Python, Python Pandas, Spark
Posted in Fabric, Spark
Posted in Fabric, Spark
Comparing performances of CSV to RDS, Parquet, and Feather file formats in R
From the previous blogpost:– CSV or alternatives? Exporting data from SQL Server data to ORC, AVRO, Parquet, Feather files and store them into Azure data lake we have created Azure blob storage, connected secure connection using Python and started uploading files…
Tagged with: arrow, benchmark, CSV, feather, file compare, Parquet, R, Spark
Posted in Uncategorized
Posted in Uncategorized
SQL vs. NoSQL for Data Science
Data come in variety of form, at different pace, and at different volume. And if all three criteria define the difference between SQL and NoSQL and there, all three are still irrelevant for data science.
My theorem is, that no matter what shape, size, frequeny, value and trustworthiness, SQL type of presenting the data is still the number one player.
Advent of 2021, Day 25 – Spark literature, documentation, courses and books
Series of Apache Spark posts: Dec 01: What is Apache Spark Dec 02: Installing Apache Spark Dec 03: Getting around CLI and WEB UI in Apache Spark Dec 04: Spark Architecture – Local and cluster mode Dec 05: Setting up Spark Cluster Dec 06: Setting up IDE Dec…
Tagged with: Azure Databricks, books, courses, Databricks, documentation, literature, Python, R, Scala, Spark, SQL
Posted in Azure Databricks, Spark, Uncategorized
Posted in Azure Databricks, Spark, Uncategorized
Advent of 2021, Day 24 – Data Visualisation with Spark
Series of Apache Spark posts: Dec 01: What is Apache Spark Dec 02: Installing Apache Spark Dec 03: Getting around CLI and WEB UI in Apache Spark Dec 04: Spark Architecture – Local and cluster mode Dec 05: Setting up Spark Cluster Dec 06: Setting up IDE Dec…
Tagged with: data, Data engineering, data visualisation, Databricks, dataframe, Python, R, Scala, Spark, SQL
Posted in Spark, Uncategorized
Posted in Spark, Uncategorized
Advent of 2021, Day 23 – Delta live tables with Azure Databricks
Series of Apache Spark posts: Dec 01: What is Apache Spark Dec 02: Installing Apache Spark Dec 03: Getting around CLI and WEB UI in Apache Spark Dec 04: Spark Architecture – Local and cluster mode Dec 05: Setting up Spark Cluster Dec 06: Setting up IDE Dec…
Tagged with: Azure, Azure Databricks, Data engineering, Databricks, Delta Live Tables, Python, Scala, Spark
Posted in Azure Databricks, Spark, Uncategorized
Posted in Azure Databricks, Spark, Uncategorized
Advent of 2021, Day 22 – Spark in Azure Databricks
Series of Apache Spark posts: Dec 01: What is Apache Spark Dec 02: Installing Apache Spark Dec 03: Getting around CLI and WEB UI in Apache Spark Dec 04: Spark Architecture – Local and cluster mode Dec 05: Setting up Spark Cluster Dec 06: Setting up IDE Dec…
Tagged with: Azure, Azure Databricks, data, Data engineering, Databricks, Python, Scala, Spark
Posted in Azure Databricks, Spark, Uncategorized
Posted in Azure Databricks, Spark, Uncategorized
Advent of 2021, Day 21 – Spark GraphX operators
Series of Apache Spark posts: Dec 01: What is Apache Spark Dec 02: Installing Apache Spark Dec 03: Getting around CLI and WEB UI in Apache Spark Dec 04: Spark Architecture – Local and cluster mode Dec 05: Setting up Spark Cluster Dec 06: Setting up IDE Dec…
Advent of 2021, Day 20 – Spark GraphX processing
Series of Apache Spark posts: Dec 01: What is Apache Spark Dec 02: Installing Apache Spark Dec 03: Getting around CLI and WEB UI in Apache Spark Dec 04: Spark Architecture – Local and cluster mode Dec 05: Setting up Spark Cluster Dec 06: Setting up IDE Dec…
 | laresbernardo on LazyMouse – R package fo… |
| laresbernardo on LazyMouse – R package fo… |
| Randomly Moving the… on LazyMouse – R package fo… | |
| Advent of 2025, Day… on Advent of 2025, Day 23 – SQL S… | |
| Advent of 2025, Day… on Advent of 2025, Day 22 – SQL S… |
TomazTsql 