Commit graph

61 commits

Author SHA1 Message Date
Watchful1
3c97f8ffcb More logging 2025-03-09 11:34:38 -07:00
Watchful1
15d574e198 Add from/to 2025-02-27 21:35:21 -08:00
Watchful1
bb25896815 More updates to overlapping users 2025-01-07 20:36:11 -08:00
Watchful1
902f142228 Add another discord flush 2024-12-11 12:16:51 -08:00
Watchful1
a4ea76a592 More logging 2024-11-28 13:07:13 -08:00
Watchful1
ce964262e3 Better link fix 2024-11-24 09:48:14 -08:00
Watchful1
f66f1011c1 Fix old lines that don't have a permalink 2024-11-24 09:44:10 -08:00
Watchful1
fba4d6de99 Fix logging 2024-11-07 21:38:53 -08:00
Watchful1
d442ef991c Merge remote-tracking branch 'origin/master' 2024-08-11 20:15:08 -07:00
Watchful1
99ae19f690 Update logging a bit 2024-08-11 20:14:56 -07:00
Watchful1
e68c8e2d11 Some more folder debugging 2024-08-10 16:50:02 -07:00
Watchful1
bda7a4c8d6 Print the full file name when it doesn't exist 2024-06-21 10:12:09 -07:00
Watchful1
94fb40cdf4 Fix null fields 2024-06-02 15:39:59 -07:00
Watchful1
fa5f6316fb Initial implentation of process_month.py 2024-05-17 21:35:00 -07:00
Watchful1
b54a2483dc Clean up overlapping users script 2024-04-24 18:11:05 -07:00
Watchful1
d67260b9ac Add ignored users list 2024-04-24 16:58:50 -07:00
Watchful1
ec977a76b2 Didn't mean to commit that 2024-03-22 19:25:53 -07:00
Watchful1
ef186b7bd7 Update merge to not overwrite scores 2024-03-07 20:41:04 -08:00
Watchful1
fe8fef722f Fix csv script for comments without a permalink 2024-02-19 21:20:37 -08:00
Watchful1
ef12dc5694 Don't crash when the dict is empty 2024-02-09 21:51:58 -08:00
Watchful1
09334829f6 Bit of cleanup for to_csv 2024-02-04 20:32:03 -08:00
Watchful1
7727934db0 Initial implementation of multiprocess recompress script 2024-01-25 22:01:17 -08:00
Watchful1
6114afb53f Fix last month of year 2024-01-07 10:45:51 -08:00
Watchful1
82966bf7f6 Add partial matching support to multiprocess script 2023-12-23 17:36:08 -08:00
Watchful1
4e140e218b Add recompress file test script 2023-11-12 16:18:04 -08:00
Watchful1
f35762e203 Add split blocks by minute 2023-11-02 21:04:03 -07:00
Watchful1
8a0256285f Update comment 2023-08-22 22:13:37 -07:00
Watchful1
d7beff9a08 Support multiple files in filter_file 2023-08-22 22:12:30 -07:00
Watchful1
0827eee152 Evidently this is a string sometimes 2023-08-22 19:43:38 -07:00
Watchful1
78c1814a60 Support empty filter 2023-08-21 21:45:58 -07:00
Watchful1
f7146593a0 Log on bad lines too 2023-08-09 19:42:56 -07:00
Watchful1
4110374fe8 Add more logging to filter file 2023-08-09 19:38:16 -07:00
Watchful1
4a50ca6605 Add overlapping users finder 2023-05-25 18:28:37 -07:00
Watchful1
897332b1d7 Update filter file with id output and filter by input list 2023-05-25 18:13:05 -07:00
Watchful1
e103298be3 Update iterate folder with the new decode method 2023-03-16 18:26:40 -07:00
Watchful1
f7286b7572 Full redesign of multiprocess 2023-03-08 17:48:59 -08:00
Watchful1
31ad7179dc Update the parse here, switch to counting instead of adding scores 2023-03-08 17:48:48 -08:00
Watchful1
1f7a3137f4 Update multiprocess to handle large numbers of output files 2023-03-06 20:37:15 -08:00
Watchful1
8dcc65abf7 Initial work on filter file 2023-03-02 18:41:50 -08:00
Watchful1
8282a5e765 Count matched lines 2023-01-30 17:53:02 -08:00
Watchful1
4e8d6c9b6b Fix to_csv chunk sizing 2023-01-30 17:05:03 -08:00
Watchful1
33b5b938c1 Change to FileHandle 2023-01-28 11:39:27 -08:00
Watchful1
87d2b22a73 Change the pool chunksize to 1 to reduce parallelization 2023-01-24 20:52:53 -08:00
Watchful1
2358bf555b Add value_list argument to take a large list of values to filter on 2023-01-24 20:51:10 -08:00
Watchful1
3415c7880e Remove filter here 2023-01-24 09:40:41 -08:00
Watchful1
cae4434c33 Bit more cleanup for combine and add count 2023-01-20 11:33:08 -08:00
Watchful1
894961c3ee Save the arguments in the status json so we don't accidentally reuse the same files for a different run 2023-01-17 22:37:25 -08:00
Watchful1
edf82d3d90 Change default encoding 2023-01-16 09:58:48 -08:00
Watchful1
1a3789c298 Work on multiprocess, change up argument format, handle comments and submissions at the same time, split the output 2023-01-12 16:46:58 -08:00
Watchful1
c4d652d0cf Update frame sizes 2022-07-17 15:56:06 -07:00