Primer on using iPython

This was more complete before I accidentally deleted most of it. For now, here's at least an introduction to what ipython's interactive shell is:

Hello iPython

Sample data to download

Homework

Have it done by Monday, April 4. You can email me at dun@stanford.edu. Send attachments, links, whatever. We won't use Github for this particular assignment.

Can you find records relating to Alvin's arrest in 2012? According to reports, it took place on June 3, 2012.
Can you confirm the ratio of innocent people stopped-and-frisked that the NY ACLU reports?

The NYPD stop and frisk data

http://www.nyc.gov/html/nypd/html/analysis_and_planning/stop_question_and_frisk_report.shtml

They also have a link for their most recent NYPD Stop Question Frisk Database File specifications

Comparing counts by the ACLU

Via the NY ACLU:

http://www.nyclu.org/content/stop-and-frisk-data

An analysis by the NYCLU revealed that innocent New Yorkers have been subjected to police stops and street interrogations more than 5 million times since 2002, and that black and Latino communities continue to be the overwhelming target of these tactics. Nearly nine out of 10 stopped-and-frisked New Yorkers have been completely innocent, according to the NYPD’s own reports:

In 2014, New Yorkers were stopped by the police 45,787 times. 37,744 were totally innocent (82 percent). 24,319 were black (53 percent). 12,489 were Latino (27 percent). 5,467 were white (12 percent).

In 2015, New Yorkers were stopped by the police 22,939 times. 18,353 were totally innocent (80 percent). 12,223 were black (54 percent). 2,567 were Latino (11 percent). 6,598 were white (29 percent).

Download the file

The two mirrors of the dataset are here:

Download the file and save it locally (so that you don't have to keep redownloading it):

import requests
url = 'http://stash.compjour.org/samples/stops-and-frisks/2015-nypd.csv'

resp = requests.get(url)
with open('2015-nypd.csv', 'w') as f:
    f.write(resp.text)

Parsing it as data:

from csv import DictReader
with open('2015-nypd.csv', 'r') as f:
    datarows = list(DictReader(f))

# count the rows
len(datarows)

Count the number of blacks (remember to check the NYPD documentation):

# quick check
for row in datarows:
    print(row['race'])

# quick check
for row in datarows:
    print(row['race'])

Finding individual cases

To Thwart a Powerful Intruder, He Let Him Open the Door - FEB. 1, 2013

But he did not stay away for long. The next day, officers spotted the suspect nearby in the Prince Street subway station and arrested him, Detective Raul Delpozo said. The police identified him as Anthony Roman, 28, and said he had been busy in the neighborhood for a month, robbing five other people. He started small, stealing $25 and Chinese food from a victim in the West Village on Dec. 17, the police said, but three days later forced a victim to withdraw $200 from an A.T.M.

import requests
url = 'http://stash.compjour.org/samples/stops-and-frisks/2013-nypd.csv'

resp = requests.get(url)
with open('2013-nypd.csv', 'w') as f:
    f.write(resp.text)

from csv import DictReader
with open('2013-nypd.csv', 'r') as f:
    datarows = list(DictReader(f))

Count the 28 year olds

xcount = 0
for row in datarows:
    if '28' == row['age'] and :
        xcount += 1
# 5269

Lesson plan for scouting the NYPD Stop and Frisk data