DEAN 690_Week 7

DEAN 690

Assignment: Week 7

Dataset

Topic: 2017 #Oscars Tweets

This dataset contains 29,000+ tweets about the 2017 Academy Awards.

Link: https://www.kaggle.com/madhurinani/oscars-2017-tweets

GMU library provides this book for FREE online version.

Natural language processing for social media

Atefeh. Farzindar author. Diana. Inkpen author. 2015

Questions

31. Explain survivorship bias?

Survivorship bias is a specific type of selection bias. This logical errors happen when making decisions, human ignore logical thinking and follow past thinking pattern. Finally, human make same past failures.

Reference Link:

Wikipedia

https://en.wikipedia.org/wiki/Survivorship_bias

Rational wiki

http://rationalwiki.org/wiki/Survivorship_bias

24. Give an explanation of collaborative filtering.

Collaborative filtering is one kind of recommendation. When searching information on the internet will show other related links. For example, when searching about specific papers on Google search, there are many links will show main information. The following line will show related article. This is one kind of collaborative filtering.

Reference

[1] J. Bobadilla, A. Hernando, F. Ortega and A. Gutiérrez, "Collaborative filtering based on significances", Information Sciences, vol. 185, no. 1, pp. 1-17, 2012.

News

KD nuggets posted an article to explain anomaly detection.

From wikipedia, the following link shows the explanation of this concept.

https://en.wikipedia.org/wiki/Anomaly_detection

Topic: Introduction to Anomaly Detection

http://www.kdnuggets.com/2017/03/datascience-introduction-anomaly-detection.html

Project Idea

Natural Language Processing for Social Media

This book mentions Introduction to Social Media Analysis / Linguistic Pre-processing\\ of Social Media Texts / Semantic analysis of social media texts, applications of social media text analysis, data collection, annotation, and evaluation. If someone work on this topic, it might be useful to brainstorm the datasets.