Practical Machine Learning, AI & Data Science on Azure ML/Server and Azure SQL/Server in R – a 5-day training course presented by Rafal Lukawiecki
This live, instructor-led course, updated for 2019, can be attended in the classroom or via Skype.
It focuses on R and the technologies of Microsoft Machine Learning Server, Azure ML, SQL Server, and Azure SQL Database, whilst teaching you everything you need to know to start using machine learning, and to apply data science for analytics.
Part A (Monday and Tuesday) introduces the most important concepts and tools. Part B (Wednesday to Friday) teaches you R and how to use it for machine learning on the Microsoft platform. Most of the course is also applicable to Python programmers, as the key ML Server libraries are the same.
Who is the course for?
Part A: Analysts, budding data scientists, database and BI developers, programmers, power users, DBAs, predictive modellers, forecasters, consultants, anyone interested in using ML for AI.
Part B: Current data scientists, ML/AI engineers, and all attendees of Part A. If you have attended a course on Machine Learning before, such as Rafal’s week-long class Practical Data Science that was offered in 2015–2017, or if you are versed in model validity, accuracy, and reliability, consider attending Part B only.
There are no prerequisites for Part A other than general ability to work with data in any form: if you have used a spreadsheet, tables, databases, or you have written a program, no matter how long ago, you will learn much from this first part of the course.
Part B expects that you have some understanding of machine learning. If you have attended a prior course on Machine Learning, or if you are versed in model validity, accuracy, and reliability, consider attending Part B only. If unsure, ask yourself these questions: can I explain the difference between cross-validation and hold-out testing? Do I know which business metrics correspond to precision and which to recall? Is model accuracy more important than reliability, and how does a boosted decision tree work. If in doubt, please attend both Parts A and B.
What you will learn
You will learn all the concepts and tools that you need to know!Rafal Lukawiecki will teach you:
everything essential to starting data science, ML, and AI projects
all fundamental concepts
how to avoid common pitfalls
how to work fast yet accurately
what is really useful and practical
what is more theoretical but still important
what hype you should be wary of.
You will be able to ask any questions related to your industry and you will get relevant, pragmatic, no-nonsense answers, helping you get ahead with your own projects.
Rafal has been delivering ML, data mining, and data science projects for customers in retail, banking, entertainment, healthcare, manufacturing, education, and government sectors for more than 10 years, and has trained more than 800 data scientists worldwide. He’s a highly-respected presenter, capable of holding your attention. Above all, you’ll be learning from a machine learning practitioner.
The training comprises 50% lectures, 30% demos and 20% labs.
You are encouraged to follow the demos on your machine, and you will be challenged to find answers to 3 larger problems during the tutorials. While they are a hands-on part of the course, if you prefer not to practice, you are welcome to use that time for additional Q&A, or to analyse your own data. We will provide you with all the necessary data sets, and we will explain what free or evaluation edition software needs to be installed to follow the course on your own laptop.
Bring your own laptop.
You will need an Azure account (even a free one) during the course. You can copy course experiments and data into your workspace for learning and for future reference after the course.
About Rafal Lukawiecki
Rafal has more than a decade of real-world machine learning experience.
As a Data Scientist at Project Botticelli Ltd, Rafal focuses on making advanced analytics and artificial intelligence easy and useful for his clients.
He can help you find valuable, meaningful patterns and statistically valid correlations using data mining and machine learning, and he is also known for his work in business intelligence, data protection, enterprise architecture, and solution delivery.
Rafal has been a popular speaker at major IT conferences since 1998, and he has had the honour of sharing keynote platforms with Bill Gates and Neil Armstrong. A natural educator, he explains complex concepts in simple terms in his enjoyably energetic style.
This course is available as live instructor-led training in the classroom and live by Skype.
To deliver the best possible training we follow the industry. The agenda and course content are subject to continuous improvement and revision without further notice.
Days 1 and 2 (Monday and Tuesday)
Part A: Introduction to Machine Learning, AI & Data Science with Azure ML
Machine Learning Fundamentals
We begin with a thorough introduction of all of the key concepts, terminology, components, and tools. Topics include:
Machine learning vs. data mining vs. artificial intelligence
Model building vs model deployment
Explorative ML vs predictive modelling
Data Driven Decisions
Tool landscape: open source R vs. Microsoft R, Python, Azure SQL Database, SQL Server, ML Server, Azure ML
Azure ML Studio vs Azure ML Services
GUI vs code-first approaches in Azure ML
Algorithms, frameworks, model validity
There are hundreds of machine learning algorithms, yet they belong to just a dozen of groups, of which 5 are in very common use. We will introduce those algorithm classes, and we will discuss some of the most often used examples in each class, while explaining which technology tools (Azure ML, SQL, or R) provide their most convenient implementation. You will also learn how to find more algorithms on the Internet and how to figure out if they are any good for real use. Topics include:
What do algorithms do?
Algorithm classes in R, Python, ML Server, Azure ML, and SSAS Data Mining
Supervised vs. unsupervised learning
Determining which algorithms/packages are good and trustworthy?
Correlation is not causation
Machine learning requires you to prepare your data into a rather unique, flat, denormalised format. While features (inputs) are always necessary, and you may need to engineer thousands of them, we do not need labels (predictive outputs) in all cases. Topics include:
Cases, observations, samples, rows, and signatures
How much data is enough?
Does big data help?
Inputs and outputs, features, labels, regressors, independent and dependent variables, factors
Data formats, discretization/quantizing vs. continuous
Azure ML data preparation and manipulation modules
Moving data around and its storage, SQL vs. NoSQL, files, data lakes, BLOBs, Data Lakes and Hadoop
Process of Data Science
The process consists of problem formulation, data preparation, modelling, validation, and deployment—in an iterative fashion. You will briefly learn about the CRISP-DM industry-standard approach but the key subject of this module will teach you how to apply the scientific method of reasoning to solve real-world business problems with machine learning and statistics. Notably, you will learn how to start projects by expressing needs as hypotheses, and how to test them. Topics include:
Start of every project: stating business needs in data science terms
Hypotheses: null vs alternative
Hypothesis testing and experiments
Evaluating test results
The problem with p-values (briefly)
Bayesian vs. frequentist approach (briefly)
Student’s t-test, Pearson chi-squared test
Iterative hypothesis refinement
Introduction to Model Building
At the heart of every project we build machine learning models! The process is simple and it follows a well-trodden path. In this module you will build your first decision tree and get it ready for validation in the next module. Topics include:
Connecting to data
Selecting features and the label
Splitting data to create a holdout
Initialising the algorithm (Two-class Boosted Decision Tree)
Training a decision tree
Scoring the holdout
Dealing with troublesome feature selection
Introduction to Model Validation
The most important aspect of any data science, artificial intelligence, and machine learning project is the iterative validation and improvement of the models. Without validation, your models cannot be reliably used. There are several tests of model validity, most importantly those that check accuracy and reliability. Topics include:
False positives vs. false negatives
Classification (confusion) matrix
Precision and recall
Balancing precision with recall vs. business goals and constraints
Introduction to lift charts and ROC curves
The bias-variance trade-off (briefly)
Deployment to Production
Although many models provide immediate, explorative value not needing any further deployment, for many others deploying them into production enables the predictive, and artificially intelligent uses. You will learn a very quick and simple way of deploying Azure ML models as a web service, which is good at earlier stages of your projects. More advanced deployment techniques are covered in the intermediate course. Topics include:
Web service vs other deployment techniques
Experiment preparation for deployment
Creating a web service
Calling a predictive web service using Request/Response API
Consuming predictive web services in Python application and in Excel
Days 3, 4 and 5 (Wednesday to Friday)
Part B: Intermediate Machine Learning in R on SQL Server and Microsoft ML Server
Working with R
There is a large number of tools that you can use with R, and we begin the day focusing on the essential ones. You will also learn how to organise your workflow. Topics include:
Why is RStudio better than RTVS 2017
R Tools for Visual Studio 2017 (please note, there is no RTVS for VS 2019)
Microsoft Machine Learning Server vs SQL Machine Learning Services (Azure and Server)
Package dependency management
Snapshots using MRAN Time Machine
Projects, files, scripts, history, version control using git
Notebooks and RMarkdown
Data Preparation in R
R uses data frames, data tables, and tibbles, amongst others, while ML Server adds XDFs and the ability to work with data stored natively in Hadoop, Spark, and SQL Server. While most data preparation should be done as close to source, preferably using SQL, you will need to learn how to perform some transformations in R. Topics include:
Data frames, tables, tibbles
Reading files and ODBC data
XDFs and connecting to data in ML Server
Scaling data access using ML Server to overcome R/Python memory and parallelism limitations
Plots and Visualisations in R
One of the strengths of R is the ease of creating accurate (and good looking!) plots. As a bare minimum you need to understand how to use the most popular visualisation package, ggplot2, and some of the built-in base functions. Topics include:
Base boxplots, histograms, scatter plots
ggplot2: grammar of graphics
Combining visualisations into layers
Surfacing R graphics in Power BI and SQL Server
Plotting big data using ML Server
Clustering, Segmentation, Anomaly Detection
Segmentation is the main application of unsupervised learning using clustering algorithms. You will also learn how to apply this technique for anomaly (outlier) detection and data preprocessing. Topics include:
Introduction to segmentation
Clustering algorithms (k-means, EM, hierarchical, and others)
Working with k-means
Preparing data for clustering, incl. categorical, non-numeric data
Informal yet practical introduction to Principal Component Analysis (PCA)
Validating cluster goodness of fit using plots and metrics
Anomaly detection with clustering, PCA and SVMs
Without doubt, classifiers are the most important, and the most often used category of machine learning algorithms, and the foundation of algorithmic data science, and of most of today’s Artificial Intelligence. We will focus on several variants of the most important classification algorithm—decision tree—while progressively interpreting the results, and improving its performance. After introducing neural networks and logistic regression we will also compare the performance of all of these classifiers on our test dataset. Topics include:
Introduction to classifiers
Two-class (binary) vs multi-class
Decision trees, forests, and boosting
Implementing simple decision trees in plain R
Visualising plain decision trees
Decision Forests and Boosting in ML Server
Overfitting (overtraining) concerns
Pruning and Complexity Penalty (CP), regularisation weight and other hyperparameters
Minimum support and the size of the tree
Avoiding overfitting through hyperparameter tuning
Implementing parallelised logistic regression on big data using ML Server
Validation of classifiers will be your key concern, because classifiers are used so often, and because their accuracy is not easy to balance with business requirements, such as restricted resources, or a required level of business performance. Building on your understanding of model validity (introduced in Part 1 of this course), you will learn how to balance an acceptable number of false positives with false negatives by using classification (confusion) matrices, metrics of precision and recall, by plotting ROC (Receiver Operating Characteristic) curves, and by measuring their business impact using profit and cost charts. Attendees have commented in the past that this is the most important module of the entire course. Topics include:
Charting precision-recall and sensitivity-specificity
Balancing precision-recall with business goals and constraints
ROC curves and lift charts in detail
Other measures of accuracy, including AUC, and F1 scores
Class imbalance problem (fraud analytics and rare event prediction)
What exactly does cross-validation tell us?
Measuring quality of cross-validation
Optimising binary classifier prediction probability thresholds for a given business target
Refining models to improve accuracy and reliability
Refining Complexity Penalty through cross-validation using caret package
Considered by some as the numerical equivalent of classifiers, regression is a large subject of its own. We will introduce its simple but a very popular form, linear regression, followed by the Generalised Linear Model and other forms of regression, and finally, the more precise, but also prone-to-overfitting, decision tree variants. Topics include:
Introduction to simple regressions in R
Linear regression (classic)
Generalised Linear Models (GLM)
Dealing with non-normal data (Gamma distribution)
Ordinal and multinomial regressions
Advice on working with (star) ratings and Likert scales
Regression decision trees and other ensemble regression algorithms
Regression as a building block of other algorithms
Unlike classifiers, regressions are easier to asses. You will learn about basic tests of classical linear regressions that are easy to perform in R, and about measuring quality of machine learning, non-linear regressions. Topics include:
Measuring linear regression quality
Homoscedasticity, multicollinearity and other concerns
Common diagnostic plots
Making prettier regression validation scatterplots in ggplot2
Measuring machine learning regression quality
R-squared (Coefficient of Determination), RMSE, MAE, RAE, RSE
Deployment to Production
If you plan on using your models for prediction, rather than just for the exploration of data, or if you want to embed them as Artificial Intelligence in your applications, you need to deploy your models to production and maintain them on an on-going basis. Since we focus on the Microsoft ML Server and SQL ML Services (both Azure Database and Server), you will learn about the powerful and fast PREDICT T-SQL statement, and other supported mechanisms for deploying your models. We will also discuss how to deploy models as a web service, using these, and other Microsoft and non-Microsoft techniques. Topics include:
What needs to be deployed, and when?
PREDICT T-SQL statement
Model storage, management and serialisation concerns
Deploying web services uses mrsdeploy and operationalisation server clusters
Consuming web services API from R
Consuming web services using Swagger and REST
On-going maintenance and model updates
Relationship to Azure ML
Please note: we reserve the right to amend the order of the modules to best suit the dynamic character of the class and to answer questions as they arise. Some subjects will only be covered if time allows, but your satisfaction is guaranteed.