ChatGPT's 'Code Interpreter': A Game-Changer in Data Science

AI now operates similar to a Graduate Research Assistant

Figure generated with MidJourney.

Introduction:

OpenAI’s ChatGPT recently released the beta version of a groundbreaking plugin known as ‘Code Interpreter’. For those unfamiliar with ChatGPT, it is an advanced language model that uses machine learning to generate human-like text based on the input it receives. The addition of ‘Code Interpreter’ to GPT-4 marks a significant milestone in the field of artificial intelligence (AI) and data science. This innovative plugin allows users to upload data files (in formats such as CSV) and then prompt ChatGPT to organize, explain, visualize, and analyze the data directly within the ChatGPT interface. This is a significant expansion of the AI model capabilities beyond natural language processing into data analysis. In this post, I will first explore the use of ‘Code Interpreter’ with an example data set from my academic research. Following this, I will share my thoughts on the significance of this development in terms of its impact on data science, user accessibility, and the future of AI.

Testing of ‘Code Interpreter’:

I decided to start by ‘kicking the tires’ on ‘Code Interpreter’ by using some old experimental data from my time in academia. This data came from a pilot study on a pain suppressant, which was a preliminary study conducted to justify a more in-depth investigation before committing to a full-scale study. The overall study has since been completed and is currently under review, with a preprint available on BioRXIV. I provided this data to ChatGPT along with a detailed explanation of the context behind the study and experimental groups.

I began by prompting ChatGPT with the relevant information surrounding the study and uploading the summarized data file. In response, ChatGPT pulled up a terminal and began coding in Python. The initial code was used to import the data and summarize the results. It correctly identified the four experimental groups in the study, demonstrating an understanding of the context. ChatGPT then produced descriptive statistics and generated a boxplot to visualize these data. It made insightful observations about the shape of the boxplot and relative differences between the groups based on their data appearances (note: does anyone know if chatgpt is actually visualizing the data or just claiming to, please comment on the answer). It made the key observation that adenosine appears to reduce the peak intensity of the sensory neurons relative to the vehicle. ChatGPT took the next logical step by testing for differences between the groups via Analysis of Variance. It performed the ANOVA, explained the results, and then contextualized them based on the experiment context I gave in the prompt. It further gave suggestions on follow-up since no significance was observed.

Further testing.

Next, I wanted ChatGPT to compare the Adenosine vs Vehicle Fracture Groups. This was likely not performed initially since the ANOVA failed to identify any significance within the groups. In response to my prompt, it compared the groups via a two-sample t-test and found no significant differences between the groups.

Challenging assumptions.

Upon reviewing the data, I did not believe parametric testing was appropriate. However, ChatGPT was making the assumption that this would be suitable. So I further challenged why it was making these assumptions. Even with prompting, it did not immediately pick up on the non-normality of the data or the outliers that existed in the boxplot it previously visualized. So I further prompted it to check the assumptions it previously made. After some prompting, ChatGPT found that parametric testing wasn’t appropriate and went to the non-parametric test I was assuming it would use ('Mann-Whitney Test’). It then found significance between the Adenosine and Vehicle groups.

Analysis and Reflections:

Upon testing the 'Code Interpreter' plugin, I found that ChatGPT not only understood the context of the data based on my description of the experiment, but it also correctly identified the key groups to compare. While the initial statistical tests it performed were not quite proper, it was able to be guided to the correct tests, which in executed and produced conclusions from. This demonstrated a level of adaptability and learning that was impressive.

In terms of its capabilities, the 'Code Interpreter' plugin operates at a level similar to that of a STEM-field graduate student. This is a significant achievement for an AI model. For comparison, traditional data analysis tools require a strong understanding of statistical methods and programming languages. Expensive statistical software such as Prism, JMP, or MiniTab often serve as alternatives for those lacking these skills. However, the 'Code Interpreter' plugin allows users to leverage the power of AI to organize, visualize, and analyze complex datasets directly, making it a more accessible and efficient tool.

One area for improvement could be the plugin's initial assumptions about the data. In my testing, it assumed that parametric testing was appropriate, which was not the case. However, with further prompting, it was able to correct this assumption and apply the appropriate non-parametric test. This shows that while the plugin is powerful, it can benefit from user input and guidance, leading to a collaborative approach to data analysis.

Overall, my experience with the 'Code Interpreter' plugin was positive. It demonstrated a strong understanding of the data and context, and with some guidance, was able to perform accurate statistical tests. I believe that with further development and refinement, this plugin could become an invaluable tool for data analysis.

What is the significance of this?

The integration of data analysis capabilities into ChatGPT has the potential to revolutionize the way data science is conducted. Currently, data analysis requires a strong understanding of statistical methods and programming languages such as R-Studio or Python. If users lack these skills, they often resort to utilizing expensive statistical software such as Prism, JMP, or MiniTab that facilitate data handling, visualization, and analysis. However, with the 'Code Interpreter' plugin, users can directly organize, visualize, and analyze complex datasets with AI. This not only streamlines the data analysis process but also makes it more efficient and accurate, as the AI can handle large volumes of data and complex computations with ease.

For example, consider a small business owner who wants to analyze customer data to improve their marketing strategies. Without a background in data science, they might struggle to make sense of the data. However, with the 'Code Interpreter' plugin, they could easily upload their customer data, prompt ChatGPT to analyze it, and receive valuable insights that could help them improve their business.

This democratization of data analysis is one of the most significant impacts of the data analysis plugin for ChatGPT. By integrating these capabilities into an AI model that can understand and generate human-like text, the plugin makes data analysis accessible to a wider audience. This could have far-reaching implications for various fields, from business to education to research, making it easier for individuals and organizations to leverage data-driven insights in their decision-making processes.

The Future of AI and Data Analysis.

The development of the 'Code Interpreter' plugin for ChatGPT provides a glimpse into the potential future of AI and data analysis. As AI models become more sophisticated and versatile, they are likely to play an increasingly central role in data science. The integration of data analysis capabilities into AI models could lead to the development of more advanced tools for predictive analytics, machine learning, and other data-driven fields.

For instance, imagine a future where AI models could not only analyze and visualize data but also predict future trends based on existing data. This could revolutionize fields like financial forecasting, climate modeling, and healthcare analytics. Furthermore, as AI becomes more accessible and user-friendly, it could lead to a greater democratization of data science, with more people able to leverage the power of data in their work and daily lives.

In education, for example, teachers could use AI tools to analyze student performance data and tailor their teaching strategies to better meet the needs of their students. In healthcare, doctors could use AI to analyze patient data and predict health risks, leading to more personalized and effective treatments.

In conclusion, the development of a data analysis plugin for ChatGPT is a significant advancement that has the potential to shape the future of AI and data science. By making data analysis more accessible and efficient, this plugin not only enhances the capabilities of ChatGPT but also opens up new possibilities for the use of AI in data-driven fields. As such, it represents a major step forward in the ongoing evolution of AI and data science.