documentcrawler

documentcrawler takes a batch of document references — DOI, title/author/year, ISBN, or keywords — and resolves and downloads full text from a configurable set of sources, open-access ones enabled by default. It ships today as a Python CLI with an optional desktop GUI and HTTP server, including FTS5 full-text search, deduplication, and metadata health auditing. It's a genuinely useful tool without a public web front end yet — turning it into a hosted webapp means building a browser UI against its existing HTTP server and standing up hosting for the Python service, which is scoped as the next step rather than a quick redeploy.

Features

• —Batch reference resolution from DOI, title/author/year, ISBN, or keyword lists
• —Multi-source retrieval — Unpaywall, OpenAlex, arXiv, PubMed Central, DOAJ by default
• —PDF full-text search (FTS5), deduplication, and metadata health auditing
• —Optional desktop GUI and HTTP server alongside the CLI

Available on Python CLI · Desktop GUI · HTTP server