|
|
8d7c6dfa2c
|
twitterchiver/relatedfetcher: fix directionality of is-not-found check
|
2021-04-01 00:38:13 +00:00 |
|
|
|
7a48bccf36
|
twitterchiver/relatedfetcher: wire bucket from flag into worker config
|
2021-04-01 00:29:46 +00:00 |
|
|
|
3d475e0efe
|
go/twitterchiver: port relatedfetcher to gocloud.dev
|
2021-04-01 00:07:41 +00:00 |
|
|
|
7003f84fb7
|
twitterchiver/related_fetcher: use tweet_mode=extended (d'oh)
|
2020-10-22 21:15:01 +00:00 |
|
|
|
a0d730cb47
|
twitterchiver/related_fetcher: use quoted_status_id instead of just quoted_status.id, sometimes only one is present
|
2020-10-22 01:28:39 +00:00 |
|
|
|
0da26f3e42
|
twitterchiver/related_fetcher: move the fetch token counter to right before we do the fetch
|
2020-10-21 01:24:30 +00:00 |
|
|
|
e49cb8eb05
|
twitterchiver/related_fetcher: up the work-cap, but introduce an absolute cap on number of tweets actually fetched
|
2020-10-20 23:30:50 +00:00 |
|
|
|
cab1df2966
|
twitterchiver/related_fetcher: treat 403 as deleted as well
|
2020-10-19 23:57:44 +00:00 |
|
|
|
e6180a6850
|
twitterchiver/related_fetcher: add separate flags for tick interval/work for each worker
|
2020-10-18 21:23:26 +01:00 |
|
|
|
297fc79db0
|
twitterchiver/related_fetcher: mark tweets as media-fetched if they have no media
|
2020-10-18 21:12:37 +01:00 |
|
|
|
50895e2aa3
|
twitterchiver/related_fetcher: fix marking tweets as done if they had no related content
|
2020-10-18 21:02:17 +01:00 |
|
|
|
cc8f24e8e1
|
twitterchiver: add relatedfetcher, to fetch referenced media and tweets
|
2020-10-18 20:44:50 +01:00 |
|
