One useful but not well-understood Python tip for data science is the use of generator expressions. Generator expressions are similar to list comprehensions, but they are more memory efficient because they do not create a new list object in memory. Instead, they generate each individual value in the expression on-the-fly, which can be especially useful when working with large datasets.
Here is an example of how to use a generator expression in Python. Note the use of parentheses () rather than square brackets [] but the syntax of the expression is otherwise the same:
# create a list of numbers from 1 to 10
numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
# use a generator expression to square each number in the list
squared_numbers = (x**2 for x in numbers)
# print the squared numbers
for num in squared_numbers:
print(num)In the code snippet above, we use a generator expression to square each number in the numbers list. This generates a generator object that behaves like any other iterable. In this example, we used it in a for loop to print each perfect square. Using a generator expression in this way is more memory efficient than creating a new list object to store the squared numbers.
Overall, generator expressions are a useful and efficient tool to have in your Python toolkit for data science tasks.
About
Einblick is an agile data science platform that provides data scientists with a collaborative workflow to swiftly explore data, build predictive models, and deploy data apps. Founded in 2020, Einblick was developed based on six years of research at MIT and Brown University. Einblick customers include Cisco, DARPA, Fuji, NetApp and USDA. Einblick is funded by Amplify Partners, Flybridge, Samsung Next, Dell Technologies Capital, and Intel Capital. For more information, please visit www.einblick.ai and follow us on LinkedIn and Twitter.