In this post, we’ll be going over two ways to perform linear regression using ordinary least squares (OLS) estimation using the statsmodels library in Python. Conveniently the two functions are OLS() and ols(). The output from statsmodels provides a number of useful diagnostic statistics and information about the model.
All of the code is available in the canvas embedded below. Otherwise, read on for a step-by-step walkthrough of how to use statsmodels to run a simple linear regression model.
Note: OLS is the standard method for fitting linear regression models. The statsmodels library, usually imported with the alias sm or smf, provides a number of different regression models.
`statsmodels.api` `sm.OLS()` example
import statsmodels.api as sm
# Create X and y dataframes
X = df[["petal_width"]]
y = df[["petal_length"]]
# Add constant according to statsmodels documentation
X_sm = sm.add_constant(X)
# Create model, fit, and print results
mod_sm = sm.OLS(y,X_sm)
res_sm = mod_sm.fit()
res_sm.summary()`statsmodels.formula.api` `smf.ols()` example
import statsmodels.formula.api as smf
# Use formula to build model, fit, and print results
mod_smf = smf.ols(formula='petal_length ~ petal_width', data=df)
res_smf = mod_smf.fit()
res_smf.summary()Setup: import packages, load data
1. Install and import seaborn, pandas, and statsmodels
2. Load and prepare your data
# Import and install packages (we'll import statsmodels later)
%pip install statsmodels
import seaborn as sns
import pandas as pd
# Load iris dataset from seaborn
iris = sns.load_dataset("iris")
iris.head()Output:
# Subset for just one species of flower
versicolor = iris[iris["species"] == "versicolor"]
# Write df to Dataframes menu
einblick.write_df("versicolor", versicolor)We subset the data for a particular flower species, versicolor, and used the einblick.write_df() to write the dataframe to the lefthand Dataframes menu. This is an optional step, but gives access to Einblick's other operators and functionality.
`statsmodels.api` `sm.OLS()` function in-depth
Now we're ready to fit our model, with our data now stored as a DataFrame, df.
Import statsmodels.api under the alias sm. This contains our first function, sm.OLS().
import statsmodels.api as sm
# Create X and y dataframes
X = df[["petal_width"]]
y = df[["petal_length"]]
# Add constant according to statsmodels documentation
X_sm = sm.add_constant(X)
# Create model, fit, and print results
mod_sm = sm.OLS(y,X_sm)
res_sm = mod_sm.fit()
res_sm.summary()Output:
3. Fit the model to your data: pass the independent and dependent variables to the OLS(y, X) function and fit the model using the fit() method.
- NOTE on
df: because we saved the dataframe earlier to Einblick's Dataframes menu, we can now use a variety of operators on the dataframe, and refer to the dataframe asdf, as we do in this instance. - NOTE on
sm.add_constant(X): in the case of theOLS()function fromstatsmodels.api, before we fit the model, we need to use thesm.add_constant(X)function, which adds a column of constants to theXdataframe, before passing that into theOLS()function. This is a structural requirement so that the ordinary least squares estimation can be performed properly. - NOTE on
sm.OLS()syntax: in the case of theOLS()function, note that the y argument comes before the X argument.
Now that the model is built and fitted, you can get the results.
4. Use the summary function for model evaluation: the fitted OLS model, res_sm, includes a summary() method that provides a detailed summary of the model fit, including the coefficient estimates, standard errors, and p-values for each feature. Based on the results, we can say that a 1-unit increase in petal width is associated with a 1.87-unit increase in petal length, with a p-value of 0.000, and a 95% confidence interval of (1.44, 2.30).
`statsmodels.formula.api` `smf.ols()` function in-depth
The next method we're using comes from statsmodels.formula.api under the alias smf. This contains our second function, smf.ols().
import statsmodels.formula.api as smf
# Use formula to build model, fit, and print results
mod_smf = smf.ols(formula='petal_length ~ petal_width', data=df)
res_smf = mod_smf.fit()
res_smf.summary()- NOTE: we need a new import statement, and that the designated OLS function is now lowercase.
- NOTE: in this case, we do not need to separate our X and y variables into separate dataframes, and we also do not need to add a column of constants to our X variables. This is due to how the
smf.ols()function works. - NOTE on
smf.ols()syntax: in the case of thesmf.ols(), the function takes in aformulaargument, which may seem familiar if you've used the programming language, R before. The argument takes in a string that represents the linear regression formula for your model. The syntax for the formula starts with the y variable name also known as your response, outcome, or dependent variable, followed by a tilde,~, and then followed by your X variable names also know as your predictor, input, or independent variables. In this case, we only have 1 X variable, so the formula looks likey ~ X. If there were multiple X variables, the formula would look likey ~ X1 + X2 + X3 + ... + Xn, where a+would separate each X variable. Thesmf.ols()function also takes in adataargument, which is our dataframe,dfin this case.
Just as above, we use the fit() and summary() methods to fit the model and get summary statistics. These methods yield the same method as above. The difference between the two functions is syntactical for you as the user, and may depend on whether or not you want to write out all of your variables in a string variable or not.
About
Einblick is an agile data science platform that provides data scientists with a collaborative workflow to swiftly explore data, build predictive models, and deploy data apps. Founded in 2020, Einblick was developed based on six years of research at MIT and Brown University. Einblick customers include Cisco, DARPA, Fuji, NetApp and USDA. Einblick is funded by Amplify Partners, Flybridge, Samsung Next, Dell Technologies Capital, and Intel Capital. For more information, please visit www.einblick.ai and follow us on LinkedIn and Twitter.