The Data Science Life Cycle is the structured process that data scientists follow — from understanding a business problem all the way to deploying and monitoring a working solution. Think of it as the foundational sequence that turns raw, chaotic data into actionable real-world value.
By following this iterative cycle, organizations ensure that their machine learning efforts are not just academic exercises but robust engineering pipelines that solve specific business needs and adapt to changing environments.
By the end of this chapter, you will be able to:
The following flowchart illustrates the 8-step iterative process. Note the decision point at Step 4, which often sends the practitioner back to cleaning or sourcing more data before proceeding to engineering and modelling.
Loading diagram...
While we follow an 8-step model, other industry-standard frameworks exist. The following table compares our cycle with the two most popular historical frameworks:
| Feature | 8-Step Internal Cycle | CRISP-DM | OSEMN |
|---|---|---|---|
| Focus | Engineering & MLOps | Business-focused process | Technical execution |
| Steps | 8 | 6 | 5 |
| Deployment | Deep focus on MLOps | Included | Often omitted |
| Monitoring | Explicit step | Implicit | Omitted |
| Best For | Modern SaaS & Production ML | Corporate BI & Strategy | Academic & Research Data Science |
Everything starts with clearly defining the problem. A vague question leads to a useless model. Good problem understanding means knowing precisely what decision the model will inform and what success looks like (e.g., accuracy vs. revenue impact).
Success Metric: A quantitative measure (like F1-score or ROI) used to determine if the model has solved the business problem.
Example: A telecom company wants to reduce customer churn. The problem: "Can we predict which customers will cancel their plan in the next 30 days based on their usage patterns?"
Once the problem is clear, you need to collect the right data. This involves working with Data Engineers to pull from databases, APIs, data warehouses, or external vendors.
What this code does: Loads customer data from a CSV file using the pandas library and displays the count of loaded records and features.
Loading code block...
Raw data is almost never ready to use. Data Wrangling (preprocessing) prepares the data for analysis by handling missing values, fixing formatting, and removing duplicates.
Data Wrangling: The process of cleaning and unifying messy and complex data sets for easy access and analysis.
What this code does: Preprocesses raw data by checking for missing values, filling missing fields in numeric columns, and removing columns with excessive missing values.
Loading code block...
Data Exploration (EDA) uses visualization and statistics to understand the data and check whether the initial questions are being answered. This is where the Data Analyst's job typically ends.
What this code does: Generates visualizations (a bar chart for churn rate and a correlation heatmap) to explore patterns and relationships in the data.
Loading code block...
This step transforms raw data into features that better represent the underlying problem to the predictive models, resulting in improved model accuracy on unseen data.
Feature Engineering: The process of using domain knowledge to create new variables (features) from raw data that help machine learning algorithms work better.
What this code does: Performs feature engineering by converting tenure to years and label-encoding categorical values into numeric format for model training.
Loading code block...
Modelling is the process of training a machine learning algorithm on the prepared data to learn patterns that can be used for predictions.
What this code does: Splits the data into training and test sets, trains a RandomForestClassifier model, and evaluates its prediction accuracy.
Loading code block...
A model creates no value until it is available to real users. Deployment makes the model available via APIs, mobile apps, or batch jobs.
MLOps (Machine Learning Operations): A set of practices that aims to deploy and maintain machine learning models in production reliably and efficiently.
After deployment, the model must be monitored. The real world changes—customer behavior shifts (Data Drift), or the relationship between features and target changes (Model Drift).
Model Drift: The degradation of model performance over time due to changes in the environment or underlying data distributions.
The Data Science Life Cycle is like Opening and Running a Successful Restaurant:
Question: What is the primary difference between Data Wrangling and Feature Engineering? Answer: Data Wrangling is focused on cleaning and preparing raw data (removing errors, missing values), while Feature Engineering is focused on creating and selecting new variables that improve the model's predictive power.
Question: Why is Step 8 (Monitoring) considered one of the most critical steps in a production environment? Answer: Because real-world data is dynamic. "Model Drift" can cause a high-performing model to become inaccurate over time as customer behavior or external conditions change. Monitoring ensures you know when to retrain.
Question: Which step typically consumes the majority (up to 80%) of a Data Scientist's time? Answer: Data Wrangling and Data Exploration. Cleaning raw data and understanding its quirks is significantly more time-consuming than the actual training of the model.