DataCrate

About DataCrate AI

What is DataCrate?

DataCrate AI is an open-source dataset provisioning platform built for the academic community. It bridges the gap between publicly available research datasets distributed via BitTorrent and the restrictive network environments found in most universities and research institutions.

Researchers and students frequently need access to large-scale datasets — ImageNet, Common Crawl, LAION, The Pile, and countless others — that are distributed as torrents. Unfortunately, institutional firewalls and network policies almost universally block peer-to-peer traffic, forcing researchers to find workarounds, use personal connections, or simply give up.

How It Works

DataCrate accepts a magnet link, fetches the dataset through a cloud-based caching layer, and provides direct HTTPS download links that work on any network — no P2P clients, no VPNs, no firewall exceptions required.

• Paste a magnet link for any publicly available research dataset
• DataCrate caches the torrent contents via secure cloud infrastructure
• Download via HTTPS — standard browser downloads that work everywhere

Who Is This For?

DataCrate is designed for university researchers, graduate students, and academic staff who need fast, reliable access to open research datasets. While currently serving the University of Calgary community, the platform is fully open source and can be deployed by any institution.

Open Source

DataCrate AI is released under an open-source license. Universities, labs, and research groups are welcome to fork, deploy, and adapt the platform for their own communities. Contributions, bug reports, and feature requests are encouraged.

• Self-hostable with Docker or bare-metal deployment
• Multi-tenant with shared API key pooling
• Minimal infrastructure requirements (Node.js, PostgreSQL, Redis)

Contact

For questions, issues, or deployment support, please open an issue on the project's GitHub repository or reach out to the maintainers through your institution.

Last updated: July 2026

Privacy Policy

Overview

DataCrate AI is committed to protecting the privacy of its users. This platform is operated as an open-source academic tool and collects only the minimum data necessary to provide dataset download services to authorized university members.

Information We Collect

• Account Information: Your university email address and a securely hashed password. Your email is used solely to verify your university affiliation and serve as your login credential.
• Dataset Requests: The magnet links you submit, dataset names, and download status. This is necessary to process and track your downloads.
• Usage Logs: Basic server logs (timestamps, request types) for operational monitoring and debugging. These do not include personal browsing activity.

Information We Do Not Collect

• We do not track your browsing activity outside this platform
• We do not use cookies for advertising or third-party analytics
• We do not sell, share, or monetize your data in any way
• We do not store your password in plain text (bcrypt hashing is used)

How We Use Your Information

• Authentication: To verify you are an authorized university member
• Service Delivery: To process magnet links and provide download links
• System Administration: To manage API key allocation and platform health

Data Storage & Security

All data is stored in a PostgreSQL database on infrastructure controlled by the platform administrators. Passwords are hashed using bcrypt with a cost factor of 12. API communication uses HTTPS encryption. JWT tokens are used for session management and expire after 7 days.

Data Retention

Account data is retained as long as your account is active. Dataset request records are kept for operational purposes. You may request account deletion by contacting an administrator, which will remove your account and all associated dataset records.

Third-Party Services

DataCrate uses TorBox as a cloud-based torrent caching service to convert magnet links into HTTPS downloads. When you submit a magnet link, it is forwarded to the TorBox API. Please refer to TorBox's own privacy policy for their data handling practices.

Open-Source Transparency

Because DataCrate is open source, you can inspect exactly what data the platform collects and how it is processed by reviewing the source code. Self-hosting institutions are responsible for their own privacy practices and should adapt this policy accordingly.

Changes to This Policy

This privacy policy may be updated as the platform evolves. Material changes will be communicated through the platform interface.

Last updated: July 2026

Terms & Conditions

1. Acceptance of Terms

By creating an account on DataCrate AI, you agree to these Terms & Conditions. If you do not agree, please do not register or use the platform.

2. Eligibility

DataCrate is available exclusively to current students, faculty, staff, and alumni of participating universities. You must register with a valid institutional email address (e.g., @ucalgary.ca or @alumni.ucalgary.ca). You are responsible for maintaining the confidentiality of your account credentials.

3. Acceptable Use

You agree to use DataCrate only for lawful academic and research purposes. You may not:

• Submit magnet links for copyrighted content that you do not have authorization to access
• Use the platform to download content unrelated to academic research or coursework
• Share your account credentials with others
• Attempt to circumvent platform access controls or rate limits
• Use the platform for any commercial purpose
• Interfere with or disrupt the platform infrastructure

4. Academic Datasets Only

DataCrate is intended for publicly available research datasets. This includes datasets published by research institutions, academic organizations, and open-data initiatives that are distributed via BitTorrent for bandwidth efficiency. Users are responsible for ensuring they have the right to access any dataset they request.

5. No Guarantees

DataCrate is provided "as is" without warranties of any kind. We do not guarantee:

• Uninterrupted or error-free service availability
• Download speed or completion time for any dataset
• Permanent storage or availability of cached datasets
• Compatibility with all magnet links or torrent formats

6. Resource Limits

The platform operates on shared infrastructure with pooled API keys. To ensure fair access for all users, administrators may impose limits on concurrent downloads, total storage, or request frequency. Abuse of shared resources may result in account suspension.

7. Account Termination

Administrators reserve the right to suspend or delete accounts that violate these terms, abuse shared resources, or are no longer affiliated with a participating institution. Users may also request voluntary account deletion at any time.

8. Limitation of Liability

DataCrate, its maintainers, and hosting institutions are not liable for any damages arising from the use or inability to use the platform, including but not limited to data loss, download failures, or service interruptions. The platform is a community tool provided at no cost.

9. Open-Source License

The DataCrate platform software is open source. Institutions deploying their own instances are responsible for establishing their own terms of service. These terms apply specifically to this hosted instance.

10. Modifications

These terms may be updated at any time. Continued use of the platform after changes constitutes acceptance of the revised terms. Material changes will be communicated through the platform interface.

Last updated: July 2026