Google’s Data Science Agent: Can It Really Do Your Job?

On March 3rd, Google officially rolled retired its Data Science Agent to astir Colab users for free. This is not thing marque caller — it was first announced successful December past year, but it is now integrated into Colab and made wide accessible.

Google says it is “The early of information study pinch Gemini”, stating: “Simply picture your study goals successful plain language, and watch your notebook return style automatically, helping accelerate your expertise to behaviour investigation and information analysis.” But is it a existent game-changer successful Data Science? What tin it really do, and what can’t it do? Is it fresh to switch information analysts and information scientists? And what does it show america astir nan early of information subject careers?

In this article, I will research these questions pinch real-world examples.

What It Can Do

The Data Science Agent is straightforward to use:

1. Open a new notebook successful Google Colab — you conscionable request a Google Account and tin usage Google Colab for free;
2. Click “Analyze files pinch Gemini” — this will unfastened nan Gemini chat model connected nan right;
3. Upload your information record and picture your extremity successful nan chat. The supplier will make a bid of tasks accordingly;
4. Click “Execute Plan”, and Gemini will commencement to constitute nan Jupyter Notebook automatically.

Data Science Agent UI (image by author)

Let’s look astatine a existent example. Here, I utilized nan dataset from nan Regression pinch an Insurance Dataset Kaggle Playground Prediction Competition (Apache 2.0 license). This dataset has 20 features, and nan extremity is to foretell nan security premium amount. It has some continuous and categorical variables pinch scenarios for illustration missing values and outliers. Therefore, it is simply a bully illustration dataset for Machine Learning practices.

Jupyter Notebook generated by nan Data Science Agent (image by author)

After moving my experiment, present are nan highlights I’ve observed from nan Data Science Agent’s performance:

• Customizable execution plan: Based connected my punctual of “Can you thief maine analyse really nan factors effect security premium amount? “, nan Data Science Agent first came up pinch a bid of 10 tasks, including data loading, information exploration, information cleaning, information wrangling, characteristic engineering, information splitting, exemplary training, exemplary optimization, exemplary evaluation, and information visualization. This is simply a beautiful modular and reasonable process of conducting exploratory information study and building a instrumentality learning model. It past asked for my confirmation and feedback earlier executing nan plan. I tried to inquire it to attraction connected Exploratory Data Analysis first, and it was capable to set nan execution scheme accordingly. This provides elasticity to customize nan scheme based connected your needs.

Initial tasks nan supplier generated (image by author)

Plan accommodation based connected feedback (image by author)

• End-to-end execution and autocorrection: After confirming nan plan, nan Data Science Agent was capable to execute nan scheme end-to-end autonomously. Whenever it encountered errors while moving Python code, it diagnosed what was incorrect and attempted to correct nan correction by itself. For example, astatine nan exemplary training step, it first ran into a DTypePromotionError correction because of including a datetime file successful training. It decided to driblet nan file successful nan adjacent effort but past sewage nan correction connection ValueError: Input X contains NaN. In its 3rd attempt, it added a simpleImputer to impute each missing values pinch nan mean of each file and yet sewage nan measurement to work.

The supplier ran into an correction and auto-corrected it (image by author)

• Interactive and iterative notebook: Since nan Data Science Agent is built into Google Colab, it populates a Jupyter Notebook arsenic it executes. This comes pinch respective advantages:
  • Real-time visibility: Firstly, you tin really watch nan Python codification moving successful existent time, including nan correction messages and warnings. The dataset I provided was a spot ample — moreover though I only kept nan first 50k rows of nan dataset for nan liking of a speedy trial — and it took astir 20 minutes to decorativeness nan exemplary optimization measurement successful nan Jupyter notebook. The notebook kept moving without timeout and I received a notification erstwhile it finished.
  • Editable code: Secondly, you tin edit nan codification connected apical of what nan supplier has built for you. This is thing intelligibly amended than nan charismatic Data Analyst GPT successful ChatGPT, which besides runs nan codification and shows nan result, but you person to transcript and paste nan codification elsewhere to make manual iterations.
  • Seamless collaboration: Lastly, having a Jupyter Notebook makes it very easy to stock your activity pinch others — now you tin collaborate pinch some AI and your teammates successful nan aforesaid environment. The supplier besides drafted step-by-step explanations and cardinal findings, making it overmuch much presentation-friendly.

Summary conception generated by nan Agent (image by author)

What It Cannot Do

We’ve talked astir its advantages; now, let’s talk immoderate missing pieces I’ve noticed for nan Data Science Agent to beryllium a existent autonomous information scientist.

• It does not modify nan Notebook based connected follow-up prompts. I mentioned that nan Jupyter Notebook situation makes it easy to iterate. In this example, aft its first execution, I noticed nan Feature Importance charts did not person nan characteristic labels. Therefore, I asked nan Agent to adhd nan labels. I assumed it would update nan Python codification straight aliases astatine slightest adhd a caller compartment pinch nan refined code. However, it simply provided maine pinch nan revised codification successful nan chat window, leaving nan existent notebook update activity to me. Similarly, erstwhile I asked it to adhd a caller conception pinch recommendations for lowering nan security premium costs, it added a markdown consequence pinch its proposal successful nan chatbot 🙁 Although copy-pasting nan codification aliases matter isn’t a large woody for me, I still consciousness disappointed – once nan notebook is generated successful its first pass, each further interactions enactment successful nan chat, conscionable for illustration ChatGPT.

My follow-up connected updating nan characteristic value floor plan (image by author)

My follow-up connected adding recommendations (image by author)

• It does not ever take nan champion information subject approach. For this regression problem, it followed a reasonable workflow – information cleaning (handling missing values and outliers), information wrangling (one-hot encoding and log transformation), characteristic engineering (adding relationship features and different caller features), and training and optimizing 3 models (Linear Regression, Random Forest, and Gradient Boosting Trees). However, erstwhile I looked into nan details, I realized not each of its operations were needfully nan champion practices. For example, it imputed missing values utilizing nan mean, which mightiness not beryllium a bully thought for very skewed information and could effect correlations and relationships betwixt variables. Also, we usually trial galore different characteristic engineering ideas and spot really they effect nan model’s performance. Therefore, while it sets up a coagulated instauration and framework, an knowledgeable information intelligence is still needed to refine nan study and modeling.

These are nan 2 main limitations regarding nan Data Science Agent’s capacity successful this experiment. But if we deliberation astir nan full information task pipeline and workflow, location are broader challenges successful applying this instrumentality to real-world projects:

• What is nan extremity of nan project? This dataset is provided by Kaggle for a playground competition. Therefore, nan task extremity is well-defined. However, a information task astatine activity could beryllium beautiful ambiguous. We often request to talk to galore stakeholders to understand nan business goal, and person galore backmost and distant to make judge we enactment connected nan correct track. This is not thing nan Data Science Agent tin grip for you. It requires a clear extremity to make its database of tasks. In different words, if you springiness it an incorrect problem statement, nan output will beryllium useless.
• How do we get nan cleanable dataset pinch documentation? Our illustration dataset is comparatively clean, pinch basal documentation. However, this usually does not hap successful nan industry. Every information intelligence aliases information expert has astir apt knowledgeable nan symptom of talking to aggregate group conscionable to find 1 information point, solving nan story of immoderate random columns pinch confusing names and putting together thousands of lines of SQL to hole nan dataset for study and modeling. This sometimes takes 50% of nan existent activity time. In that case, nan Data Science Agent tin only thief pinch nan commencement of nan different 50% of nan activity (so possibly 10 to 20%).

Who Are nan Target Users

With nan pros and cons successful mind, who are nan target users of nan Data Science Agent? Or who will use nan astir from this caller AI tool? Here are my thoughts:

1. Aspiring information scientists. Data Science is still a basking abstraction pinch tons of beginners starting each day. Given that nan supplier “understands” nan modular process and basal concepts well, it tin supply invaluable guidance to those conscionable getting started, mounting up a awesome model and explaining nan techniques pinch moving code. For example, galore group thin to study from participating successful Kaggle competitions. Just for illustration what I did here, they tin inquire nan Data Science Agent to make an first notebook, past excavation into each measurement to understand why nan supplier does definite things and what tin beryllium improved.
2. People pinch clear information questions but constricted coding skills. The cardinal requirements present are 1. nan problem is intelligibly defined and 2. nan information task is modular (not arsenic analyzable arsenic optimizing a predictive exemplary pinch 20 columns).. Let maine springiness you immoderate scenarios:
   • Many researchers request to tally analyses connected nan datasets they collected. They usually person a information mobility intelligibly defined, which makes it easier for nan Data Science Agent to assist. Moreover, researchers usually person a bully knowing of nan basal statistical methods but mightiness not beryllium arsenic proficient successful coding. So nan Agent tin prevention them nan clip of penning code, meanwhile, nan researchers tin judge nan correctness of nan methods AI used. This is nan aforesaid usage lawsuit Google mentioned erstwhile it first introduced nan Data Science Agent: “For example, pinch nan thief of Data Science Agent, a intelligence astatine Lawrence Berkeley National Laboratory moving connected a world tropical wetland methane emissions task has estimated their study and processing clip was reduced from 1 week to 5 minutes.”
   • Product managers often request to do immoderate basal study themselves — they person to make data-driven decisions. They cognize their questions good (and often nan imaginable answers), and they tin propulsion immoderate information from soul BI devices aliases pinch nan thief of engineers. For example, they mightiness want to analyse nan relationship betwixt 2 metrics aliases understand nan inclination of a clip series. In that case, nan Data Science Agent tin thief them behaviour nan study pinch nan problem discourse and information they provided.

Can It Replace Data Analysts and Data Scientists Yet?

We yet travel to nan mobility that each information intelligence aliases expert cares astir nan most: Is it fresh to switch america yet?

The short reply is “No”. There are still awesome blockers for nan Data Science Agent to beryllium a existent information intelligence — it is missing nan capabilities of modifying nan Jupyter Notebook based connected follow-up questions, it still requires personification pinch coagulated information subject knowledge to audit nan methods and make manual iterations, and it needs a clear information problem connection pinch cleanable and well-documented datasets.

However, AI is simply a fast-evolving abstraction pinch important improvements constantly. Just looking astatine wherever it came from and wherever it stands now, present are immoderate very important lessons for information professionals to enactment competitive:

1. AI is simply a instrumentality that greatly improves productivity. Instead of worrying astir being replaced by AI, it is amended to clasp nan benefits it brings and study really it tin amended your activity efficiency. Don’t consciousness blameworthy if you usage it to constitute basal codification — nary 1 remembers each nan numpy and pandas syntax and scikit-learn models 🙂 Coding is simply a instrumentality to complete analyzable statistical study quickly, and AI is simply a caller instrumentality to prevention you moreover much time.
2. If your activity is mostly repetitive tasks, past you are astatine risk. It is very clear that these AI agents are getting amended and amended astatine automating modular and basal information tasks. If your occupation coming is mostly making basal visualizations, building modular dashboards, aliases doing elemental regression analysis, past nan time of AI automating your occupation mightiness travel sooner than you expected.

Being a domain master and a bully communicator will group you apart. To make nan AI devices work, you request to understand your domain good and beryllium capable to pass and construe nan business knowledge and problems to some your stakeholders and nan AI tools. When it comes to instrumentality learning, we ever opportunity “Garbage in, garbage out”. It is nan aforesaid for an AI-assisted information project.

Featured image generated by nan writer pinch Dall-E