Tag Archives: Python Open Lab

Python Open Labs: April 23, 2018

Today was the last Python Open Lab of the semester – congrats to all of the students who have made it this far and picked up skills in a new programming language!

Over the course of the semester, we’ve been learning the basics of Python: how to initialize lists, create dictionaries, iterate through items, and define functions and classes.

The students wanted to see how programming could be applied to a specific problem and how it could be used to analyze existing information or data. I chose to design the last lesson around data visualization. We particularly focused on how to create visualizations using the seaborn library.

The seaborn library is a visualization library based off of matplotlib. It also has the capability to store datasets as dataframes, similar as to how pandas may store an external file. I have recently been exploring seaborn and already find it a very flexible and intuitive library. Borrowing concepts from a tutorial via DataCamp, we were able to create some very beautiful visualizations using only a few lines of code.

Check out some things we were able to make below!

a swarm plot displaying customer tip amounts

a facet grid displaying total bill amounts based on varying aspects of gender and dining time

a colored heat map displaying information related to airplane flights

Students really enjoyed using seaborn and some were even able to apply it to their own datasets. Lots of people were specifically fans of the swarm plots.

Yang Rui (left) and Elena Dubova (right) learning to master seaborn

If you’d like to follow the lesson for today’s class more closely, please click here for step-by-step instructions and enjoy coding things up in your favorite text editor.

Python has become a really popular programming language in the past years. I am glad to see more and more people taking the initiative to learn it and can’t wait to see the amazing challenges my students will take on in the future!

Navie Narula

Python Open Labs: April 9, 2018

We spent class today reviewing functions and how they work in Python. Students were given problem statements and were asked to write functions to return the correct output. We went over multiple problems, and I’ll step through one in this blog post today.

Imagine that you are given two inputs in the form strings: Jewels and Stones.

Jewels contain unique characters, while Stones do not.

Here is what an example of what these inputs might look like:

Jewels = “aA”

Stones = “aAAbbbb”

Students were asked to write a function to count the number of Jewels present in Stones. In the example above, the output would be 3 given that “a” and “A” are Jewels and that there is 1 of “a” and 2 of “A” in Stones.

Students in the class understood that functions must start with a definition and contain a return statement. What was more difficult to come up with was the syntax used to solve the problem within the actual function itself.

Students eventually came up with the idea to initialize a value to store a result and simply loop through the Stones string to check how many Jewels appear in the Stones input.

Here is how one might solve the problem in code using Python:

def countJewelsinStones(Jewels, Stones):

>>>>count = 0

>>>>for s in Stones:

>>>>>>>>if s in Jewels:

>>>>>>>>>>>count += 1

>>>>return count

We can see that the approach is not only simple, but also uses concepts we’ve reviewed in previous lessons such as conditionals and for loops. I am excited to see that many students in the class were able to solve this problem with little assistance and can’t wait to see what they accomplish next!

Navie Narula

Python Open Labs: March 26, 2018

Over the past few weeks, students have been learning how to iterate through items – whether they may be in strings, lists, tuples, or dictionaries. Students have mainly been using for loops to grab the value of each item, and lots of progress has been in regards to writing them with little to no instruction.

In this blog post, I wanted to go over two different ways to write for loops that I presented in class. One method uses the loop to take on the literal value of an item, whereas the other method uses the loop to take on the index of the item.

Let’s say that we have a list:

lst = [1, 2, 3, 4, 5, 6]

If I wanted to iterate through the list such that my iterator takes on the literal value of the list, then I would write my for loop like this:

for item in lst:

>>>>print(item)

The output of this program would simply be:

1
2
3
4
5
6

If I wanted my for loop to instead take on the index value of each item within the list, then I would write my for loop like this:

for item in range(len(lst)):

>>>>print(item)

The output of the program would now be:

0
1
2
3
4
5

The only difference between this for loop and the previous one is the additional use of the keywords range and len, which allow the iterator to take on an item’s index value.

Note that using the for loop structure with these keywords also allow you to take on the literal value of the list when you index the list within the loop.

Here is example of what that looks like:

for item in range(len(lst)):

>>>>print(item, lst[item])

The output of this program is:

0 1
1 2
2 3
3 4
4 5
5 6

Writing for loops either way is acceptable, but it’s important to know which one might be most relevant to your program. If I were simply looking to add one to every item in the list and print its output, using the for loop without range or len is just fine. If I had multiple lists to iterate over that all happen to be the same length, I might want to incorporate the keywords to save time and efficiency in my program.

For instance, let’s say I had two lists that were of the same length.

animals = [“dog”, “bird”, “horse”]

nums = [2, 11, 19]

I would only need to iterate through one list to grab the values from both by using range and len.

Here is what that looks like:

for item in range(len(animals)):

>>>>print(animals[item], nums[item])

My output would look like:

dog 2
bird 11
horse 19

I hope you found this blog post about for loop structure and output helpful and are confident enough to know which ones to use in your own programs!

Navie Narula

Python Open Lab Nov 28: Blog Style!

Hello all,

Due to some complex scheduling issues, I am posting here the material we would have covered in lab tomorrow. Please feel free to contact me for any questions (data@library.columbia.edu). Enjoy!

Python Objects and Classes (cont’d!)

Self: 

What is the self variable in Python?

The self variable represents the instance of the object itself. Unlike most object-oriented languages that pass the instance of an object as a hidden parameter to the methods defined on an object; Python does not. It must be explicitly declared. All methods in python, including some special methods like initializer, have self.

In other words, the self  variable refers to the object which invokes the method. When you create new object the self parameter in the __init__  method is automatically set to reference the object you have just created.

 

More theory…do’s and don’ts of Self:

You use self when:

  1. Defining an instance method. It is passed automatically as the first parameter when you call a method on an instance, and it is the instance on which the method was called.
  2. Referencing a class or instance attribute from inside an instance method. Use it you want to call a method or access a name (variable) on the instance the method was called on, from inside that method.

You don’t use self when:

  1. You call an instance method normally. For example if you input [instance = MyClass()], you call [MyClass.my_method] as [instance.my_method(some_var)] not as [instance.my_method(self, some_var)].
  2. You reference a class attribute from outside an instance method but inside the class definition.

Let’s try an example!

First, we must create an object from class:

Input:

1       p1 – Person(‘anna’)   # here we have created a new person object called p1
2       print(p1.whoami())
3       print(p1.name)

 

1      You are anna
2      anna
As discussed in lab, it is bad practice to give access to your data fields outside of the class itself. Let’s see how we can hide data fields: To hide data fields, first you have to define private data fields. In Python, this can be done by using two leading underscores (__). Moreover, a private method can also be defined using two leading underscores.

Here’s an example I created on Jupyter notebook. If you’d like, I can email you the notebook as lines are colour formatted and the spaces match up to the correct inputs (as you know, your code will be affected by the spacing).

 

Expected Output:

 

 

Now, I’d like to show you if it’s possible to access  __balance  data field outside of the class.

Input:

 

As you can see, now __balance  is not accessible outside the class.

What questions do you have at this point? Does this match up with the aforementioned theoretical concept of self/hidden self data? Would you like more practice?

Fall 2017 Python Open Lab Week 3

October 10, 2017

Week 3’s lab was intense! We started with list methods, where we left off last week and went through the following:

          list.append(x): Add an item to the end of the list.

          list.extend(L): Extend the list by appending all the items in the given list.

          list.insert(i, x): Insert an item at a given position.

          list.remove(x): Remove the first item from the list whose value is x  (it will come up as an              error if there is no such item).

          list.pop([i]): Remove the item at the given position in the list, and return it. If no index is                specified, a.pop() removes and returns the last item in the list.

          list.index(x): Return the index in the list of the first item whose value is x (it will come up                as an error if there is no such item).

list.count(x): Return the number of times x appears in the list.

list.sort(cmp=None, key=None, reverse=False): Sort the items of the list in place.

list.reverse(): Reverse the elements of the list, in place.

 

Here is an example that uses most of the list methods 

>>> a = [66.25, 333, 333, 1, 1234.5]
>>> print(a.count(333), a.count(66.25), a.count(‘x’))
2 1 0
>>> a.insert(2, 1)
>>> a.append(333)
>>> a
[66.25, 333, -1, 333, 1, 1234.5, 333]
>>> a.index(333)
1
>>> a.remove(333)
>>> a
[66.25, -1, 333, 1, 1234.5, 333]
>>> a.reverse()
>>> a
[333, 1234.5, 1, 333, -1, 66.25]
>>> a.sort()
>>> a
[-1, 1, 66.25, 333, 333, 1234.5]
>>> a.pop()
1234.5
>>> a
[-1, 1, 66.25, 333, 333]

 

We then introduced the Python Dictionary:

Python Dictionary

In the Python dictionary, each key is separated from its value by a colon (:), the items are separated by commas, and the whole thing is enclosed in curly braces.

An empty dictionary without any items is written with just two curly braces, like this: {}. Keys within a dictionary must be a data type such as strings, numbers, or tuples.

Example:

dict = {‘Name’: ‘Michael’, ‘Age’: 7, ‘Class’: ‘First’}

print “dict[‘Name’]: “, dict[‘Name’]
print “dict[‘Age’]: “, dict[‘Age’]

When we run this code, it produces the following result:

dict[‘Name’]:  Michael
dict[‘Age’]:  7

 

And finally, we very briefly touched upon string methods and boolean methods – by no means did we cover all the material we intended (as you see below) – but will pick up in Week 4 with string methods first!

 

String Methods

The string data type has multiple methods. Here are all of the methods of list objects:

str.upper()  – Making Strings uppercase

str.lower() – Making Strings lowercase

str.join() method will concatenate two strings, but in a way that passes one string through another.

str.split() method returns a list of strings that are separated by whitespace if no other parameter is given.

str.replace() method can take an original string and return an updated string with some replacement.

Boolean methods:

Method True if
str.isalnum() If string consists of only alpha-numeric values
str.isalpha() If string consists of only alphabets
str.islower() If string consists of only lower-case  values
str.isnumeric() String consists of only numeric characters
str.isspace() String consists of only whitespace characters
str.istitle() String is in title case
str.isupper() String’s alphabetic characters are all upper case

Example:

>>> string = “Hello”

>>> string.upper()

‘HELLO’

>>> string.lower()

‘hello’

>>> string = “Hello,world”

>>> string.split(“,”)

[‘Hello’, ‘world’]

>>> string[::-1]

‘dlrow,olleH’

>>> len(string)

11

>>> string1 = “Hello”

>>> string2 = “World”

>>> string1+string2

‘HelloWorld’

Hope this material is helpful and inspiring. Please don’t hesitate to contact us for any clarifications. And as always, if you have any requests for material to cover in our labs, please email or contact us!

 

Fall 2017 Python Open Lab Week 2

Week 2: October 3rd

This week we started with a brief review of the basics from week 1 and the starter kit. We continued on with Data Structures and worked through Lists, Tuples and Dictionaries. These concepts were easy to approach and we went over many practice examples on the way. Towards the end of the session, we introduced the different types of lists that one can use in Python as well as list methods. List methods, however, are quite complex and we only got through .append and .extend in great detail.

Next week we will continue with list methods!

Please comment or email us for any questions. Below is the worksheet we used for this week:

 

Python Open Lab Week II

Outline and reference:

  1. Summary from Open Lab I – Starter Kit
  2. Data Structures:

Lists:

Lists are what they seem – a list of values. Each one of them is numbered, starting from zero – the first one is numbered zero, the second 1, the third 2, etc. You can remove values from the list, and add new values to the end. Example: Your many cats’ names.

Tuples:

Tuples are just like lists, but you can’t change their values. The values that you give it first up, are the values that you are stuck with for the rest of the program. Again, each value is numbered starting from zero, for easy reference. Example: the names of the months of the year.

Dictionaries:

Dictionaries are similar to what their name suggests – a dictionary. In a dictionary, you have an ‘index’ of words, and for each of them a definition. In python, the word is called a ‘key’, and the definition a ‘value’. The values in a dictionary aren’t numbered – tare similar to what their name suggests – a dictionary. In a dictionary, you have an ‘index’ of words, and for each of them a definition. In python, the word is called a ‘key’, and the definition a ‘value’. The values in a dictionary aren’t numbered – they aren’t in any specific order, either – the key does the same thing. You can add, remove, and modify the values in dictionaries. Example: telephone book.

Lists:

  • Indexing
  • Slicing
  • List operations (concat, etc.)
  • Multiplying
  • “In” operator

*Question: Given a year, month, day, print as follows:

21st July, 1991

Methods:

  • Append
  • Extend
  • Count
  • Index
  • Insert
  • Pop
  • Remove
  • Reverse
  • Sort/Sorted

Fall 2017 Python Open Lab Week 1

Week 1 September 26
In week 1 of our Python Open lab, we introduced the Python Starter Kit and went over Python basics such as expressions, variables and statements, floats, statements, integers, strings, booleans, and control flow statements. The class was a mix of students with little to no experience with Python, to a few advanced users, to those with some knowledge of basic concepts looking to strengthen their skillset.
We moved at a slower pace in this first session to ensure students grasped the basic concepts needed to continue on their programming journey with confidence.
We hope that anyone with outstanding questions contact us, and anyone thinking of attending sign up through the DSC website. All are welcome and we will ensure you are caught up with the relevant material we covered in the past!
Next week, we will start with a review of the starter kit and continue with lists.

Python Open Labs

In the penultimate session of Python Open Labs we had a brief review of csv concepts and XML parsing with BeautifulSoup Library.

A boiler plate code for practise can be obtained below from the Google Drive link, filed under Session-19 folder.

All of the course slides and examples are made available on: https://goo.gl/YP0c2E

Next week on April 21, I will going over the basic concepts and some practise problems from the python concepts that we looked into !

Python Open Labs – CSV Files

As we move towards the end of Spring semester, having covered most of the basics in Python, the recent sessions have been focusing on introducing python modules requested by attendees.

Last week had a second session on web-scraping with BeautifulSoup, I have updated the practice code for the same in Session-17 folder of the google drive link mentioned below.

This week, on April 7, 2017 I introduced the Python CSV module for reading and writing data from csv files. A very easy module primarily for reading CSV data, it requires the user to understand only a few of the details. The relevant sample code and a practise csv file can be found on the google drive link below, under Session-18 folder.

All of the course slides and examples are made available on: https://goo.gl/YP0c2E

Next week’s topic is XML file parsing !

See you next Friday from 1:30 PM – 3:30 PM at DSSC (Room – 215), Lehman Library at Columbia SIPA !

Python Open Labs – Web Scraping

For those who have been following this blog series, sorry for a late post on the updates about Python Open Labs.

Last week we covered some basics about web scraping with python, but before I start let me make a customary disclaimer.

Make sure that any of the websites that you want to scrape have granted you the required permissions to do so. Make sure you are not violating any terms of use by doing so.

So, getting along with the updates. In a nutshell web scraping can be described as a way of extracting useful relevant information from web pages i.e html pages. This can be abstracted into following steps:

  1. Downloading the web page content (user urllib or requests module in python)
  2. View page source in a web browser to examine the html structure of web page and locate information of interest for your task at hand
  3. Try to figure out the html structuring such as class, id, html tag etc that will help your python script locate the information.
  4. Use the beautifulsoup python module to parse and reach as close as possible to the relevant information in the html page structure and then extract the information using string methods.

The steps 2 – 4 go hand in hand, i.e one helps you build more upon the other. For example, the more you understand about the html structure surrounding your page the more specific inputs you can provide to beautifulsoup methods to extract out the information.

For the previous I have uploaded the sample python files with commented code lines on the Google Drive link mentioned below which you can access under Session – 16 folder. Make sure you work through those. Doubts, queries, feedbacks are always welcome 🙂

All of the course slides and examples are made available on: https://goo.gl/YP0c2E

We will be continuing with the web scraping lecture on march 31, 2017 after which I will also upload a comprehensive document with some additional relevant sources and more interesting code.

Happy Scraping !

See you next Friday from 1:30 PM – 3:30 PM at DSSC (Room – 215), Lehman Library at Columbia SIPA !