Category Archives: Statistics & Data

Student work, take a look!

Congratulations to the graduating class of 2014! We know you might not all have had the chance to see your fellow students work from the Video Storytelling class, so here is a link to a web site created by your class mates. http://who-is-y.com/ These videos are some examples of a the great work you all created while here. We know you will do so much more in the future. Congratulations again!

(note: the videos are currently locked because the students are working to see if they can be published with other sources first. So, please enjoy the site for now, we will update with a password ASAP. At least you have an idea of what you can look forward to!) 🙂

Government Information 101: Part 3, Finding Gov Info

How to Find Government Information

So, you've learned the basics of US government and you know the main government information resources. Now you're actually looking for a government document. Where do you start? Ask yourself:

  • At what level of government do I need this information?
    • Federal, state, local (county or city)
  • What government branch or agency might be involved? Stated another way: who would have a need to gather this data or information?
  • If you're looking for data or demographic information, you should also ask: what level of geography do I need this data for? The answer could be: nation, region, state, county/borough, city, PUMA, zip code, neighborhood, census tract, or census block. For more on geographies for NYC neighborhoods, see this handout.

Gov Info Search Example

Here's an example. I'm looking for the number of traffic accident fatalities in New York State in 2010.

At what government level do I need this?

Well, I'm looking for the total number of traffic accident deaths in New York State, so that's a state-level question–though it may also be reported at the federal level.

What government agency would gather that information?

It turns out that this is reported by two different agencies. Traffic deaths are reported both by the Department of Transportation's (DOT) Bureau of Transportation Statistics (there is a GIS-run state statistic finder), and the CDC's National Center for Health Statistics (NCHS), which lists data on Deaths & Mortality, including causes of death. Under causes of death, the NCHS reports traffic-accident deaths under the broader category of "Accidents (unintentional injuries)," which is subdivided into "Transport accidents," which is further subdivided into motor vehicle accidents, other land transport accidents, and water/air/space accidents. The NCHS also provides direct data through CDC Wonder, which allows you to choose variables such as age, and specify the specific type of transport accidents (for instance, you can choose to include pedestrian, motorcycle, car, bus, railway, and streetcar accidents, but exclude animal and water transport accidents). I chose ages 18+ (to capture all adults).

What level of geography do I need?

In addition to the total state number of deaths, I'd also like to see the total deaths for each NYC borough. Since the five NY boroughs are counties, in CDC Wonder I selected data for New York State, at the county level. I found that the total number of traffic-accident deaths in NYS in 2010 (according to the criteria I selected) was 1,242.

Note: Finding Recent Statistics/Data

It's not always easy to find data for the past year, or even the year preceding that. Data takes time to gather, clean up, and make public, and often you may find that the most recently available statistics are for 2-5 years ago.

Government Information 101: Part 2, Gov Resources

Uncle Sam With Magnifying GlassIn the previous post, I discussed the basics of U.S. government (its levels, branches, and how to approach finding government information). Now I'll list some of the best places to find government information, both by government branch and with a list of popular documents. First, here are a few places to start a search for government information, particularly when you're not sure what branch or agency might have collected the information that you need:

Government Resources, by Branch

Information for many federal government agencies is centrally located in FDsys (the Federal Digital System).

Legislative Resources

Executive Resources

Judicial Resources

Other Popular Government Areas & Documents

Government Information 101: Part 1, U.S. Gov Basics

New to the U.S., or just need to brush-up on some government basics? Here's a quick run-down of U.S. government at various levels and how to find government information. (I'll be writing two follow-up posts that tell where to find primary source information for each branch of government, and show an example of a gov info search.) First, here is a U.S. government chart and three basic guides to the federal government:

Levels & Branches of Government

The United States has several levels of government: federal, state, and local (county/city). Federal is the overarching national government, which has three branches:

 

NYC Borough Counties:

Each state has its own government which echoes the federal government in its three-branch organization. The Legislative branch is composed of the state senate and house (or assembly), the Executive branch is composed of the Governor and typically many departments (structure varies by state), and the Judicial branch is composed of the state court system, headed by the state supreme court. There are different local government offices at the county and the municipal (city/town) level. Typically, cities follow a similar pattern to federal and state government, with a Mayor acting as the chief executive officer of a city. Usually, a county is larger than and encompasses many cities and towns. However, New York City is made up of five boroughs, which are each actually counties (listed at right).

What Topics are Covered by Government Documents?

The U.S. government covers a surprising number of programs, and thus data and information is available on nearly any topic you can think of, including:

  • Data & Statistics
  • Health & Vital Statistics
  • Politics & Law
  • Business & Economy
  • Treaties
  • Scientific & Medical Research
  • Technology
  • Intellectual Property (patents, trademarks, copyright)
  • Historical Events (primary resources)
  • Consumer Information
  • Grants
  • Starting & Financing a Small Business
  • Recipes & Nutrition
  • Maps
  • Education, Teacher Resources
  • Rules & Regulations
  • …and you can use Browse Topics or the US Gov Portal to find more!

Some of the few topics that aren't covered well in U.S. gov info include:

  • Literature
  • Arts
    • Exceptions: arts funding (NEA, NEH), nonprofit finances, & some gov-funded museum exhibits are covered; in addition, Whistler briefly worked for the Coastal Survey and produced engravings
  • Music
    • Exception: music copyright and copyright cases are well covered

CNTS Dataset on Domestic Conflict Events (International Coverage)

In addition to the wealth of data that is publicly available through federal, state, or local government sources, at Columbia you have access to a number of other rich datasets. You can access these datasets through the Numeric Data Catalog and the Spatial Data Catalog, which are maintained by the DSSC Data Service at Lehman Library. (And if you’ve never checked out the page on NYC data sources, you should do so now!)

One example of these interesting datasets is the Cross-National Time-Series Data Archive. This resource was started in 1968, and some of its data goes back to 1815. Of primary interest is that this dataset records occurrences of domestic conflict events (defined as: General Strikes, Purges, Government Crises, Riots, Assassinations, Anti-Government Demonstrations, Guerilla Warfare, and Revolutions) for many countries, going back to 1919. This dataset has released a new feature beginning with the 2011 data, in which each of these listed events is also linked to a relevant New York Times story. You can find frequencies for the events in the main file <CNTSDATA.xls>, and the links to related NYT stories in another file <2012 Edition LINKS.xls or 2013 Edition LINKS.xls>.

The CNTS FAQ page provides information on variables, countries included, the codebook, and more:
http://www.databanksinternational.com/71.html

Before downloading this data, you must read ICPSR’s Responsible Use Statement, available here:
http://www.icpsr.umich.edu/icpsrweb/content/datamanagement/lifecycle/access.html

ICPSR’s guide to data citation:
http://www.icpsr.umich.edu/icpsrweb/content/datamanagement/citations.html
 
For any questions about using this or other datasets, contact the DSSC Data Service at: dssc.data@columbia.edu or 212-854-6012
 

2014 Data Visualization Contest

We are excited to announce the 2nd annual Data Visualization Contest for the Journalism Library!  Submit your data visualizations to journalism@library.columbia.edu by 5pm on Monday, May 5th to be included. (Only CU Journalism students are eligible for this contest.)

The winning entry will be printed as a poster and displayed with your name (and the project URL, where applicable) in the Journalism Library.  The winner will also receive a $25 Starbucks giftcard, have your project showcased on the Libraries’ Digital Center Projects webpage, and the opportunity to archive your work in Columbia University Libraries’ Academic Commons. This is not only a chance to get your work reviewed by a panel of faculty and library experts; it’s also an opportunity for greater exposure for your work. Last year’s winner, Jefferson Mok, went on to have his project picked up by CNN!

Contest Rules:

  1. Submissions must use publicly available data. (“Data” is broadly defined and can include video, audio, photo.)
  2. Submissions must be received no later than 5pm on May 5th.
  3. You may use previously submitted class work!
  4. Submissions may be in any file format, including URLs and other dynamic/interactive digital projects (a static image or screenshot will be required for the poster in the J-Library).
  5. Please do not use your name in the filename. Instead, in your submission email include the following:  your full name, graduation year, a title for your creation, and the data source(s) used.

All submissions will be judged based on accurate use of data and originality in aesthetic presentation; panel of judges includes Journalism Librarian, Starr Hoffman; Data Services Coordinator, Ashley Jester; and J-School Professor, Susan McGregor.

The winner will be announced at the J-School Innovation Showcase before graduation.  Please email journalism@library.columbia.edu with any questions. We look forward to your submissions; good luck!

Finding Building & Property Information

Lately, we've had many questions in the Journalism Library about finding building histories, owner names, code violations, market value, and similar things. Here's a round-up of the most useful tools to find that information! (Photo below is a 1902 panorama of NYC from the Library of Congress.)

Panoramaic photograph of New York City, taken in 1902.

Dept. of Buildings: BIS (Building Information System)
http://a810-bisweb.nyc.gov/bisweb/bsqpm01.jsp

Lists actions, complaints, violations, alterations, certificates of occupancy, # buildings on lot, tax block/lot # (which you need for ACRIS, listed below), landmark status, zoning documents, etc. (Under “actions,” “NB” records are permits for new buildings.) When multiple buildings are on one lot, click on the number of buildings to access building-specific records. You can also use the BIS's "Building on my Block" to find properties in a block (or community district) that are new buildings or have had alterations. Good for getting a feel for what’s going on with property development in a given area.

Related: Building Permit Search, 1900 – 1986 (Office of Metropolitan History) http://www.metrohistory.com/searchfront.htm

Property Tax Bill & Information System (NYC Finance: Property) http://nycprop.nyc.gov/nycproperty/nynav/jsp/selectbbl.jsp

Market value history, building classification (what type of building it is), owner & billing address, tax charges, exemptions, abatements & any credits. You can find additional information (like number of stories or units, building dimensions, etc.) under Property Tax Benefit Information.

ACRIS (Automated City Register Information System; NYC Finance: Property) http://a836-acris.nyc.gov/CP/

Find mortgages, deeds, and other property documents. “Det” provides detailed information about the document (including the names and address of involved parties); “img” shows the document. Related NYC Property tools include the FY 2014 Final Assessment Roll for NYC and the Comparable Values for Condos & Coops.

NYC Housing and Vacancy Survey (Census Bureau) http://www.census.gov/housing/nychvs/

Includes information on rent-vs-own status, rent-to-income ratios, estimated value, and structural conditions (such as condition of exterior or interior walls).

Housing Code Violation Data (Dept. of Housing Preservation & Development)

HPD Online (find violations by address): https://webapps.hpdnyc.org/HPDonline/provide_address.aspx

New Residential Construction: http://www.census.gov/construction/nrc/

Reports from the U.S. Census on nation-wide permits for new residential construction.

U.S. Housing Market Conditions (HUD, U.S. Dept. of Housing and Urban Development): http://www.huduser.org/portal/ushmc/home.html

Housing market data and analysis, home affordability, and related information. Available at the national, regional, metro, and local levels (some NYC-area data).

Landmark Designation Reports (only recent designations are here): http://www.nyc.gov/html/lpc/html/forms/reports.shtml

Landmark Designations, 1965 to present. (For older reports, contact the Landmark Commission directly.)

NYC Rent Guidelines Board Reports:
http://www.housingnyc.com/html/research/cresearch.html

Reports on housing supply, income & affordability, changes to rent stabilized housing stock, and more.

American FactFinder (Census Bureau): http://factfinder2.census.gov/

Two particularly relevant tables (both available at the census tract level) are:

  • Selected Economic Characteristics (DP03)
  • Selected Housing Characteristics (DP04)

Additional Information

For more information on NYC buildings, property, and real estate, consult with librarians at Columbia's Avery Art and Architecture Library.

Avery Library Subject Guides on related topics:

Congratulations to Data Viz Contest Winner, Jefferson Mok!

Jefferson Mok - The NRA Network

We are excited to share the work of recent J-School graduate, Jefferson Mok – the winner of the Journalism Library Decorate Our Wall data visualization contest!  In "The NRA Network," his use of NRA data, illustrating the wide distribution of funds around the country toward various programs, was a wonderfully innovative, and cleverly illustrated, way to share funding activities by this particular organization.

Please stop by the Journalism Library to see the poster of his data visualization.  Congratulations again to Jefferson!