Polars FASTER than Pandas (libraries)

Discussion regarding the spreadsheet functionality of Bet Angel.
Post Reply
User avatar
ilovepizza82
Posts: 490
Joined: Thu Nov 02, 2017 3:41 pm
Location: Sewers
Contact:

Hi

Performed a quick test and these are the results:
Pandas read time: 0.149984 seconds
Polars read time: 0.100036 seconds

The test was very simple: Column A consisiting of 10.000 random numbers.

Tried with 100.000 random numbers (100k rows):
Pandas read time: 1.744550 seconds
Polars read time: 1.027678 seconds

Some people report 15gb big file was 10x faster with polars than pandas.

You can use my python code to calculate yourself:

Code: Select all

import time
import pandas as pd
import polars as pl

file = 'r:blu.xlsx'

# Pandas
start = time.time()
df_pandas = pd.read_excel(file)
end = time.time()
pandas_time = end - start

# Polars
start = time.time()
df_polars = pl.read_excel(file)
end = time.time()
polars_time = end - start

print(f'Pandas read time: {pandas_time:.6f} seconds')
print(f'Polars read time: {polars_time:.6f} seconds')
User avatar
ilovepizza82
Posts: 490
Joined: Thu Nov 02, 2017 3:41 pm
Location: Sewers
Contact:

lol forgot to add the libraries are Python libraries, sorry about that.
Those who code in Python know how important they are when you deal with Excel spreadsheets.
User avatar
ShaunWhite
Posts: 9731
Joined: Sat Sep 03, 2016 3:42 am

Excel is slowing you down.

SQL : 5,075,438 "rows" of data each with 18 columns
Screenshot_2.jpg
You do not have the required permissions to view the files attached to this post.
User avatar
ilovepizza82
Posts: 490
Joined: Thu Nov 02, 2017 3:41 pm
Location: Sewers
Contact:

LOL wow !
Post Reply

Return to “Bet Angel - Spreadsheet / Excel chat”