Merge branch 'rename-data-files' into add-code

# Conflicts:
#	data/LICENSE

Author: hpasapp

data/LICENSE

@@ -1,8 +1,9 @@
-There are two datasets in this folder:
+There are four dataset files in this folder:
 
-- airlines_raw_merged.csv.bz2 and airlines_500_mergedb.csv.bz2 are derived from Kaggle 'Customer Support on Twitter' dataset, https://www.kaggle.com/thoughtvector/customer-support-on-twitter,
+- airlines_raw.csv.bz2 and airlines_processed.csv.bz2 are derived from Kaggle 'Customer Support on Twitter' dataset,
+https://www.kaggle.com/thoughtvector/customer-support-on-twitter,
 which is available under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) license,
 https://creativecommons.org/licenses/by-nc-sa/4.0/
 
-- askubuntu_raw_mergedb.csv.bz2 and askubuntu_raw_mergedb are derived from Stack Exchange Data Dump, https://archive.org/details/stackexchange, which is provided
-under a Creative Common Attribution-ShareAlike 3.0 Unported (CC BY-SA 3.0) license, https://creativecommons.org/licenses/by-sa/3.0/
+- askubuntu_raw.csv.bz2 and askubuntu_processed.csv.bz2 are derived from Stack Exchange Data Dump, https://archive.org/details/stackexchange,
+which is provided under a Creative Common Attribution-ShareAlike 3.0 Unported (CC BY-SA 3.0) license, https://creativecommons.org/licenses/by-sa/3.0/

data/README.md

@@ -1,16 +1,23 @@
 Data
 ----
 
+Training vs test data
+---------------------
+
+We train on all data, without labels. We use the labels in order to evaluate the resulting clusters.
+
 Twitter Airlines Customer Support
 ---------------------------------
 
 The data is available in two version:
 
-- raw: minimal redaction (company and customer twitter id), no preprocessing: [airlines_raw_merged.csv.bz2](airlines_raw_merged.csv.bz2)
-- redacted, and preprocessed:  [airlines_500_mergedb.csv.bz2](airlines_500_mergedb.csv.bz2)
+- raw: minimal redaction (company and customer twitter id), no preprocessing: [airlines_raw.csv.bz2](airlines_raw.csv.bz2)
+- redacted, and preprocessed:  [airlines_processed.csv.bz2](airlines_processed.csv.bz2)
+
+492 examples are labeled using annotators. The remaining examples are labeled `UNK`.
 
 AskUbuntu
 ---------
 
-- raw, no preprocessing: [askubuntu_raw_mergedb.csv.bz2](askubuntu_raw_mergedb.csv.bz2)
-- preprocessed: [askubuntu_merged_d.csv.bz2](askubuntu_merged_d.csv.bz2)
+- raw, no preprocessing: [askubuntu_raw.csv.bz2](askubuntu_raw.csv.bz2)
+- preprocessed: [askubuntu_processed.csv.bz2](askubuntu_processed.csv.bz2)