llms as an alternative to statistical regression models
An interesting use of LLMs that I’m seeing recently is how non-technical teams are using it as an alternative to traditional statistical ML models. A recent example I saw was how one of the non-technical teams at work used a no-code LLM integration to create a price index prediction model for one of the flows/products.
Yes, the alternative could’ve been a more sophisticated regression model built by an ML/Data Science team, but do they need that level of accuracy?
This made me curious, so I ran the classic experiment of predicting house prices through an LLM vs. a linear regression model, and the results were surprising!
I used GPT-5.2 as the LLM (cheaper than Anthropic models) vs. Python sklearn’s implementation of linear regression. The Real estate price prediction Kaggle dataset was used (414 records) with an 80/20 split for the linear regression. For the LLM, the test size was the same as the linear regression model (83 records, 20% of 414), and we attached few-shot examples in each prompt. The experiment code and results can be found in mohammedamarnah/llm-vs-regression. I ran the experiment multiple times, and it cost $0.63 for around 390k input tokens for all the runs.
LLM’s mean absolute percentage error was 16.08% compared to the 17.18% on the linear regression model! Here are the metrics that were calculated:
| Metric | Linear Reg | LLM (gpt-5.2) |
|---|---|---|
| Mean Absolute Error (MAE) | 5.31 | 5.67 |
| Root Mean Square Error (RMSE) | 7.31 | 8.17 |
| Coefficient of determination (R2) | 0.68 | 0.60 |
| Mean absolute percentage error (MAPE) | 17.18% | 16.08% |
It is important to note that these results are not deterministic. Every run will produce slightly different results. Also, most applications are either more complicated than a house price prediction problem or contain more complex data (e.g., non-numeric) that requires proper encoding. The experiment was also run on a relatively small dataset.
But consider this: a non-technical team wants to build such a prediction model. They use an automation tool like Zapier ($0-$20/month) or Activepieces ($0 if self-hosting) and build this pipeline without the need for any ML/DS support, and it completely fits the need. The LLM can understand all kinds of data. It can accept aggregated data from multiple sources, and it doesn’t require any data cleaning or complicated mathematical models, just natural language. Do you think it’s worth it?