Filter-V2

With the use of the Chrome extension, we have observed a significant increase in the number of requests made to the filter, often reaching a thousand within a 30-minute span.

Improvements:

• Implemented caching for frequently accessed data, especially for results of expensive queries.
• Utilized Django's caching framework to reduce database load during high traffic periods.
• Various caching backends, such as Redis, are implemented for enhanced performance.

Since clients can set probabilities for the class result, reproducing results from false positives will require us to know the class thresholds for the request at the time it was made. Save in request history table (DB). Request history is only kept for 30 days.

Improve synchronization between the filter service and the Discord bot for recording request history.

Previously, both the filter service and the Discord bot were independently creating request history entries, resulting in duplicate entries. Furthermore, the fields in these entries did not match, leading to inconsistencies in the displayed data in the request history views.

To address this issue, we have implemented a solution. The filter service now includes the request ID it generates, and the Discord bot reads this ID to update the corresponding record with additional information, rather than creating a new entry in the database.

This ensures better alignment between the filter service and the Discord bot in recording request history, eliminating duplicate entries and ensuring consistent data in the request history views.

At a high level: Implement custom model training per client.
 
TL;DR you can add custom data per licence and we train to block that type of content. It should be noted that training requires premium as you would need to select data from discord server logs to add to to the clients' custom training data 

As per PB-89 we do not store the custom training data on FS but in DB, and we do not do version as this issue originally went towards.

Implement DeepSense filter

We now return a service errorcode if the licence does not exist.

The bot already had an error code for this - I also tweaked the message, used a different code.