DATA FOUNDRY · ACADEMIC PREPRINTS
Clean machine-readable ArXiv preprints in machine learning and CS.
Verified partition of 9,000 ArXiv submissions across artificial intelligence, machine learning, and computer science. Features clean abstracts, LaTeX tags, and primary classifications.
COMMERCIAL ACCESS
Verified Parquet archives + Claude MCP integration
- ✓ Verified Data Foundry Parquet partitions (zstd)
- ✓ Model Context Protocol (MCP) tool integration
- ✓ Metering: Inquire for Access
- ✓ Deterministic quality gate & hash verification
- ✓ Contractual commercial rights clearance
Quotes reviewed by systems engineering within 1 business day
COLUMN SPECIFICATIONS
Typed relational schema.
Columnar Parquet & DuckDB ready.
| Column Name | Data Type | Description |
|---|---|---|
arxiv_id |
VARCHAR(24) | Canonical ArXiv paper identifier |
submitted_date |
DATE | First submission date |
primary_category |
VARCHAR(16) | Primary subject category (e.g. cs.AI, cs.LG) |
title |
VARCHAR(512) | Sanitized paper title |
abstract |
TEXT | Full paper abstract with clean formula representation |
SYSTEM STATUS & CAPABILITY BOUNDARIES
Engineering reality.
Verified operational features vs planned protocol capabilities.
Verified Operational Today
- ✓ Apache Parquet Partitions: Columnar archives (Zstandard compressed) with typed relational schemas.
- ✓ Zero-Copy Local Analytics: In-memory SQL exploration via DuckDB, Python Polars, and Arrow.
- ✓ Cryptographic Deduplication: SHA-256 release manifests and seen-keys ledger preventing record drift.
- ✓ Statutory Rights Clearance: 100% license-cleared provenance with statutory citations.
- ✓ Technical Quote Intake: Systems engineering review and schema sample verification.
Pilot & In-Specification Capabilities
- ◌ Remote MCP Streaming Adapter:
foundry_search_arxiv_preprintstool endpoint available under enterprise pilot agreements. - ◌ Automated Micro-Settlement: Agent-to-agent procurement and escrow via EXCHANGER (active protocol specification).
- ◌ Multi-Region Edge Replicas: Global Cloudflare R2 bucket replication currently in private validation.
VERIFIED SAMPLE RECORD
Inspect the raw payload.
Zero mocked fields.
{
"arxiv_id": "2401.02938v2",
"title": "DeepSeek LLM: Scaling Open-Source Language Models with Longtermism",
"primary_category": "cs.CL",
"submitted_date": "2024-01-05"
}
AGENT INTERFACE & MCP TOOL CONTRACT
Plug directly into Claude Code, Codex, and autonomous workflows.
Deterministic data retrieval.
Machine-Readable Contract
Direct dataset extraction is delivered via high-throughput columnar Apache Parquet. For autonomous assistants, ELYRAON provides standard Model Context Protocol (MCP) tool schemas and REST integration adapters with signed SHA-256 integrity receipts.
Adapter identifier: foundry_search_arxiv_preprints
Pricing status: Inquire for Access
Active MCP Endpoint: https://elyraon.com.co/agents/mcp (Host to live structural verification tools; metered dataset adapters available in scoped pilots).
{"tool": "foundry_search_arxiv_preprints", "arguments": {"category": "cs.AI"}}
MACHINE-READABLE SPECIFICATIONS
Structured metadata for autonomous agents.
Zero scrapers, zero hallucinations.
{
"entity_id": "https://elyraon.com.co/datasets/arxiv-ai-cs-preprints/#dataset",
"catalog_id": "https://elyraon.com.co/datasets/#catalog",
"canonical_url": "https://elyraon.com.co/datasets/arxiv-ai-cs-preprints/",
"primary_format": "application/vnd.apache.parquet",
"compression": "zstd",
"ingestion_pipeline": "arxiv_metadata.py",
"mcp_tool_name": "foundry_search_arxiv_preprints",
"license": "Creative Commons CC0 & Non-exclusive Distribution License",
"duckdb_quickstart": "SELECT * FROM read_parquet('arxiv-ai-cs-preprints.parquet') LIMIT 5;"
}
WHY ELYRAON DATA FOUNDRY
Architectural contrast.
Built for engineering reality.
Structures messy LaTeX and newline formatting into clean UTF-8 strings ready for embedding models and automated summarization.
Conventional Data Feeds
- Undetected socket drops during volatility
- Messy unstructured CSV/JSON with broken parsers
- Ambiguous copyright terms creating lawsuit liability
- Expensive multi-thousand dollar rigid contracts
ELYRAON Data Foundry
- Continuous quality gate reconciliation
- Clean, typed, content-addressed Apache Parquet
- 100% license-cleared provenance with statutory citations
- Clear request-quote pricing and cryptographic delivery verification
RELATED DATASETS & AGENT PLATFORM
Connected intelligence feeds.
Cross-domain discovery and autonomous infrastructure.
TECHNICAL FREQUENTLY ASKED QUESTIONS
Clear answers.
Engineering boundaries.
What is the verified record count in Data Foundry?
9,000 verified preprint submissions in `arxiv_submissions`.
Which categories are covered?
cs.AI (Artificial Intelligence), cs.LG (Machine Learning), and cs.CL (Computation and Language).
Can AI research agents search abstracts semantically?
Yes. Columnar Parquet enables instant regex, keyword, and embedding queries.
Can we license ongoing daily feeds?
Yes. Nightly ArXiv mailing ingestion can be configured via ELYRAON Foundry.
How is access quoted?
Submit an inquiry with your research volume.