Upload a CSV to Snowflake (or any Pandas dataframe)

Paul Yang - September 25th, 2022

The Snowflake web-based data loader is incredibly painful and clunky to use in my opinion, and so I have just been using Einblick to throw data onto there — whether a demo CSV someone wants to use, or automating movement from our Hubspot CRM to Snowflake, I’ve run this quick script dozens of times this year.

You can upload a CSV and drop it directly in as a dataframe. Or finish your whole notebook and then just plug the results dataframe in and write it to database.

!pip install sqlalchemy  #https://docs.sqlalchemy.org/en/14/core/connections.html
!pip install snowflake.sqlalchemy  #this example uses Snowflake
from sqlalchemy import create_engine
from snowflake.sqlalchemy import URL

#set your connection to Snowflake -- you can use any DB here with SQLAlchemy
engine = create_engine(URL(
    account = 'xx',
    user = 'xx',
    password = 'xx',
    database = 'xx',
    schema = 'xx',
    warehouse = 'xx',
    role='xx'
))
con=engine.connect()

chunksize = 10000 ##how many rows to send at a time
tablename = 'xx' ##what table to write into 

df.to_sql(name=tablename, con=engine.connect(), if_exists='append', index=False, index_label=None, chunksize=chunksize)

Start using Einblick

Pull all your data sources together, and build actionable insights on a single unified platform.

  • All connectors
  • Unlimited teammates
  • All operators