Commits

Branch selector

 main

User selector

All users

All time

Commit History

Commits on Jan 25, 2023

• Adding tasks.py from ccs3.

  wpietri

  committed
  4acc37e

  Copy full SHA for 4acc37e

Commits on Nov 9, 2022

• Merge pull request #222 from CybersecurityForDemocracy/crowdtangle-raw

  Show description for d63a22a
  wpietri

  authored
  d63a22a

  Copy full SHA for d63a22a

Commits on Nov 4, 2022

• Adding a separate, simpler tool to fetch raw Crowdtangle output.

  wpietri

  committed
  5216aef

  Copy full SHA for 5216aef

Commits on Oct 23, 2022

• Adding --raw-output-path, which lets you save raw crowdtangle output to timestamped files.

  Show description for e2347ce
  wpietri

  committed
  e2347ce

  Copy full SHA for e2347ce

Commits on Apr 12, 2022

• Bump pillow from 9.0.0 to 9.0.1 (#220)

  Show description for aa15a2c
  dependabot[bot]

  authored
  aa15a2c

  Copy full SHA for aa15a2c

Commits on Mar 6, 2022

• add argument for ID attribute name to dedupe_records_with_same_id_by_max_updated_field so it can be used to dedup records with ID in different field name (#219)

  macpd

  authored
  b2ae6d4

  Copy full SHA for b2ae6d4

Commits on Mar 4, 2022

• dedupe posts, before writing to DB, with same ID by max updated (ie only most recent version of post) (#218)

  Show description for b8049b6
  macpd

  authored
  b8049b6

  Copy full SHA for b8049b6

Commits on Mar 3, 2022

• Add simplified, independent version of ad renderer from internal code base (#217)

  tlauinger

  authored
  dae2b82

  Copy full SHA for dae2b82

Commits on Feb 8, 2022

• add impressions_estimate to impressions, demo_impression_results, and region_impression_results tables (#216)

  macpd

  authored
  8104820

  Copy full SHA for 8104820

Commits on Jan 28, 2022

• Increase all min_spend, max_spend, and spend_estimate columns (impressions, demo_impression_results, and region_impression_results) size capacity (#215)

  Show description for fdc0578
  macpd

  authored
  fdc0578

  Copy full SHA for fdc0578

• add impression.spend_estimate column to schema (which is present in actual DB) (#214)

  macpd

  authored
  0832ff0

  Copy full SHA for 0832ff0

Commits on Jan 24, 2022

• Bump pillow from 8.3.2 to 9.0.0 (#213)

  Show description for 3a732e9
  dependabot[bot]

  authored
  3a732e9

  Copy full SHA for 3a732e9

Commits on Sep 8, 2021

• Bump pillow from 8.2.0 to 8.3.2 (#212)

  Show description for 7ccc7bd
  dependabot[bot]

  authored
  7ccc7bd

  Copy full SHA for 7ccc7bd

Commits on Aug 18, 2021

• Allow config to not have MAX_REQUESTS or ARCHIVE_ADVERTISERS_FILE. factor max_request check to diff function to simplify handling no limit (#211)

  macpd

  authored
  b7f0c61

  Copy full SHA for b7f0c61

Commits on Jul 28, 2021

• add retry to config_utils.get_database_connection in case of psycopg2.OperationalError (#210)

  Show description for 5064daf
  macpd

  authored
  5064daf

  Copy full SHA for 5064daf

• use facebook-sdk package from HEAD (instead of pinned past version)

  macpd

  authored
  5354ac5

  Copy full SHA for 5354ac5

Commits on Jul 26, 2021

• bump minet version to min 0.53.3 to address API connection aborted error from crowdtangle API (#209)

  macpd

  authored
  9a0c809

  Copy full SHA for 9a0c809

Commits on Jul 9, 2021

• make crowdtangle API rate limit (queries per minute) configurable per dashboard (#208)

  Show description for d3632d0
  macpd

  authored
  d3632d0

  Copy full SHA for d3632d0

Commits on Jun 30, 2021

• track min/max post updated field seen. log query info, and min/max post updated seen on CrowdTangleError (#207)

  Show description for d38d3be
  macpd

  authored
  d38d3be

  Copy full SHA for d38d3be

Commits on Jun 18, 2021

• if deadlock exception raised retry database writes of crowdtangle data upto 3 times (with random exponential backoff up to 1min) (#206)

  Show description for 2a0aa13
  macpd

  authored
  2a0aa13

  Copy full SHA for 2a0aa13

Commits on Jun 15, 2021

• add --dry_run option where output is not written to DB and is printed to stdout instead (#205)

  macpd

  authored
  48fb6b7

  Copy full SHA for 48fb6b7

• upgrade minet version to 0.52.10, and refactor code to handle interface/datasturct changes (#202)

  Show description for aa9da11
  macpd

  authored
  aa9da11

  Copy full SHA for aa9da11

• log internal dashboard_id in fetch query info log messages (#204)

  macpd

  authored
  a6e98c9

  Copy full SHA for a6e98c9

Commits on Jun 14, 2021

• Bump pillow from 8.1.1 to 8.2.0 (#203)

  Show description for 21414e9
  dependabot[bot]

  authored
  21414e9

  Copy full SHA for 21414e9

Commits on Jun 4, 2021

• clean up unused ctor args in WriteCrowdTangleResultsToDatabase (#201)

  Show description for 90a0dfa
  macpd

  authored
  90a0dfa

  Copy full SHA for 90a0dfa

• Handle multiple crowdtangle dashboards in same pipeline, fetch and process in parallel (#200)

  Show description for e643dd2
  macpd

  authored
  e643dd2

  Copy full SHA for e643dd2

Commits on Jun 3, 2021

• attach dashboard id to encapsulated post (#199)

  Show description for 7cbcb75
  macpd

  authored
  7cbcb75

  Copy full SHA for 7cbcb75

Commits on Jun 1, 2021

• update some pypi deps (#198)

  Show description for dfefac6
  macpd

  authored
  dfefac6

  Copy full SHA for dfefac6

• do not upload blob if something exists at that path (#197)

  Show description for fb410cd
  macpd

  authored
  fb410cd

  Copy full SHA for fb410cd

Commits on May 25, 2021

• add config vars for database schema and paths for ssl cert files (#196)

  macpd

  authored
  cfb674c

  Copy full SHA for cfb674c

Commits on May 17, 2021

• Add language_code to DB schema, and populate with languageCode from API response. (#195)

  Show description for 4050d67
  macpd

  authored
  4050d67

  Copy full SHA for 4050d67

Commits on Apr 30, 2021

• Bump rsa from 4.6 to 4.7 (#194)

  Show description for c7666b1
  dependabot[bot]

  authored
  c7666b1

  Copy full SHA for c7666b1

Commits on Apr 19, 2021

• add column present in fb_us_ads_2020 DB not in fb_global_ads DB

  macpd

  authored
  4c509e3

  Copy full SHA for 4c509e3

Commits on Apr 16, 2021

• connect to database for each batch of interactions instead of reusing single connection. (#193)

  Show description for 1d0f1c6
  macpd

  authored
  1d0f1c6

  Copy full SHA for 1d0f1c6

Commits on Apr 6, 2021

• connect to DB for each interaction so that dropped connections do not crash program (#192)

  macpd

  authored
  66bbf3c

  Copy full SHA for 66bbf3c

Pagination

Previous

Next