This database is PolicyMap, which we do have in our library. Here is a description: http://library.lasalle.edu/policymap

This video shows how to download data: Tables Overview

If you google search for data, include dataset in the search

Some collections of Data sets

http://archive.ics.uci.edu/ml/

https://www.statcrunch.com/5.0/shareddata.php

http://shawndra.pbworks.com/datasets

Corgis – from Virginia Tech, some data prep done to make attractive for classes - https://think.cs.vt.edu/corgis/csv/

KD Nuggets http://www.kdnuggets.com/2011/02/free-public-datasets.html and http://www.kdnuggets.com/datasets/

This one was sent to me by a student, but it looks like a lot of the datasets may be more advanced than what we are ready for: http://www.cs.cmu.edu/~guestrin/Class/10701/projects.html

Contest data at: http://www.sigkdd.org/kddcup/index.php Many of these, especially recent ones, may be too advanced (e.g. text mining)

11 datasets from Berkeley (geared toward stats courses) http://www.stat.berkeley.edu/users/statlabs/labs.html

Data wrangling – some links to datasets http://www.datawrangling.com/some-datasets-available-on-the-web

Also

Edinburgh potential class projects http://www.inf.ed.ac.uk/teaching/courses/dme/html/datasets0405.html Many of these may be too advanced (e.g. text mining)

Datamob list of datasets http://datamob.org/datasets/show/facebook-dataset (seems like many of these are actually APIs instead of datasets)

The US Census bureau is a pretty decent source of data. A lot of other government agencies make data available as well.

Robert McGinley (student) noted that a lot of cities are now making a lot of data available. Examples:

http://www.opendataphilly.org/

https://data.cityofchicago.org/

https://data.sfgov.org/

Info on countries, you can choose the countries and (up to 10) columns to include: http://hdrstats.undp.org/en/tables/ (watch out for missing values, also watch out that it is saving as .xls but is actually html. Excel can probably open despite the error message)

Search for Social Science data on the web: http://3stages.org/idata/

Research Pipeline, list of sources (can’t say that most of the sources I looked at looked like what we need) http://www.researchpipeline.com/mediawiki/index.php?title=Main_Page

(In Spanish) list of sources http://www.webmining.cl/2011/01/15-datasets-gratis-para-data-mining/

The Interuniversity Consortium for Political and Social Research (ICPSR) provides access to an extensive collection of downloadable data. Some of the data is in difficult to access format though, requiring programming to extract the data.

If you are interested in sports there is a lot of data available from various sources.

Heritage Health Care has put data up for people to investigate

The Netflix contest data is probably still available (don’t know what format it is in)

Info Chimps (some free) http://www.infochimps.com/datasets

Teens and Gaming: http://pewinternet.org/Shared-Content/Data-Sets/2008/February-2008--Teen-Gaming-and-Civic-Engagement.aspx

Chess contest: http://www.kaggle.com/c/ChessRatings2 (2011). Likely to be too hard.

Illinois Statistical Analysis Center http://www.icjia.org/public/sac/index.cfm (and http://www.icjia.org/public/sac/index.cfm?metasection=forms&metapage=publications ) has county level data for crime and related variables for many years. However, some limitations – data for different variables (columns) tend to be in different spreadsheets, making data preparation a lot of work. Plus, Illinois is not very interesting, as Chicago (Cook County) is larger and different from most of the state.

National Longitudinal Survey of Youth

https://www.nlsinfo.org/investigator/pages/search.jsp?s=NLSY97

National Bureau of Economic Research – a lot of these are in formats for statistical software such as SAS, SPSS etc

http://www.nber.org/data/

The World Bank has a lot of available data related to countries:

http://data.worldbank.org

Mark Guzdial’s list of data sources:

http://swiki.cc.gatech.edu/compfreak/5

Possible:

Really mixed contents:

http://www.augmentedintel.com/wordpress/index.php/augmented-intel-free-online-analytics-applications-for-corporate-intelligence/searchable-list-of-public-data-mining-datasets/

Amazon – I think you need to register with them, and use their tools to extract?

http://aws.amazon.com/public-data-sets/

Apparently there is something called archive.org, but I haven’t been able to tell if there is useful info there – looks like it might be a lot of text and video sources

?

http://lib.stat.cmu.edu/datasets/

Music with Last.fm (but API I think)

http://www.last.fm/api

processed, but tarred and gzipped

http://www.cs.cornell.edu/~shuochen/lme/data_page.html

SIGCSE 2014 - entries

Arizona State University has a Big Data program:

www.public.asu.edu/~ynsilva/BigData

Resources available

· Complete works of Shakesphere (text mining – too hard)

· Weather station data

TPCH benchmark database – available Dietrich ASU (I don’t know what this is about)

Kaggle runs contests involving data. an (evolving) set of data is available there:

www.kaggle.com

IMDB may be available as a database – available for download

Teradata university program includes datasets (we may need to sign up)

Reportedly, VT (Virginia Tech) providing Real Time Web as a resource

http://research.cs.vt.edu/vtspaces/realtimeweb/

or

goo.gl/io6Iwt

Client libraries
Curated gallery

Libraries can function without internet connectivity (internal data cache)

They offer a number of data sources

US Weather forecasts
Reddit Link Shares
Stock Prices
Generator to generate new datasets

Ruth Anderson, Michael Ernst, Robert Ordonez, Paul Pham, Steven Wolfram

http://tinyurl.com/dataprogramming

dataprog@googlegroups.com

Seems like might have good resources

Reported Potential Real available Datasets:

Coming Fall 2014 – wolfram.com/technology/guide/LoadOnDemand/CuratedData/
Electoral-vote.com (Tanenbaum) (posts .csv files)
Earthquake and volcano data
gov
NOAA – weather
Census
NCAA tournament data
Divvy bikes – clean data – visualization challenge
cityofchicago.com?
Quantified self – how people walk (from iphone accelerometers)
Stock prices?
IMDB
Project Guttenberg (text)
Mars rover
CDC
Twitch plays Pokeman
Music sharing comments
Open Data (cities) – e.g. is every house with in ____ of a fire station (arrays)
Oceanographic data

Purple America

Data visualization, real data

colored states and counting not red or blue but a shade based on % of vote

Olympic buzz on Facebook by country by sport

notes that there is a Plug-in for Eclipse that allows students to access their own Facebook Data

Enron e-mail data set (would likely be text mining, though social link analysis could also be possible)

http://www.cs.cmu.edu/~enron/

Flights in USA – on time late etc

https://www.transtats.bts.gov/DL_SelectFields.asp?Table_ID=236&DB_Short_Name=On-Time

Movie daily box office (time series for one movie at a time, so not that easy to work with)

http://www.the-numbers.com

Non Free – there’s no reason to pay for data given how much is available free

Some of the databases available from the Healthcare Cost and Utilization Project (HCUP) http://www.ahrq.gov/research/data/hcup/index.html

Nationwide Inpatient Sample (NIS)
The Nationwide Inpatient Sample (NIS) is the largest publicly available all-payer hospital inpatient care database in the United States. Researchers and policymakers use NIS data to identify, track, and analyze trends in health care utilization, access, charges, quality, and outcomes. ($350; $50 for students)

Nationwide Emergency Department Sample (NEDS)
The Nationwide Emergency Department Sample (NEDS) captures information on emergency department (ED) visits that do not result in an admission as well as ED visits that result in an admission to the same hospital. The NEDS was created to enable analyses of ED utilization patterns and to support public health professionals, administrators, policymakers, and clinicians in their decision-making regarding this critical source of care. (HUGE)