This is a proposal to replace the storage backend for chat history. I am requesting comments and suggestions before doing a lot of working.
I am proposing that chat history's backend be changed from text files in a directory structure on the filesystem to sqlite db. SQLite is a minimal sized library capable of storing millions of records and retrieving them efficiently. Many chat clients already use sqlite as storage for locally-stored chat histories (Skype for example.) This would make retrieving chat history much faster then it currently is on large datasets (10+ years of chat history) which currently stall the viewer in several places.
It also opens up the viewer to better tools like the ability to search chat logs for key phrases or quickly bounce around chat history to certain dates and times. Useful for CSR purposes, etc. It is true that some OS's already index chat logs and that this is possible to do outside of the viewer, but it is not purpose built to do it.
Migration code would be a necessity. This would be best run during initialization and slurp up chat logs into the db. I do not propose removing the old chat logs as part of this. A marker file would be added to tell the viewer chat logs have already been migrated.
As earlier stated, Operating System file indexing would be affected. I don't believe Windows or MacOS Spotlight grok sqlite. Both do, however, have apis for indexing application data in smarter ways.