Using the ArcticDB DataFrame database for tick storage
ArcticDB is an embedded, serverless database engine, tailored for integration with pandas and the Python data science ecosystem. It’s used for the storage, retrieval, and processing of petabyte-scale data in DataFrame format. It uses common object storage solutions such as S3-compatible storage systems and Azure Blob Storage or local storage. It can efficiently store a 20-year historical record of over 400,000 distinct securities under a single symbol with sub-second retrieval. In ArcticDB, each symbol is treated as an independent entity without data overlap. The engine operates independently of any additional infrastructure, requiring only a functional Python environment and object storage access.
ArcticDB was built by Man Group and has demonstrated its capacity for enterprise-level deployment in some of the world’s foremost organizations. The library is slated for integration into Bloomberg’s BQuant platform...
