🎯 Purpose
This repository contains complete reverse-engineered documentation for ibi device databases, enabling the data recovery community to build tools for recovering family photos and videos from ibi devices.
Context: SanDisk officially discontinued ibi support on August 31, 2024, leaving thousands of families unable to access their photos. This open-source toolkit provides the free alternative to expensive commercial recovery services.
📋 What’s Included
1. Complete Schema Documentation
schema_documentation.md
- Full SQLite schema with all tables, relationships, and field descriptions
api_specification.json
- Machine-readable API spec for automated tool development
reference_implementation.py
- Working Python parser demonstrating all major operations
2. Analysis & Insights
metadata_strategy.md
- What metadata to preserve vs. what to filter out
metadata_strategy.md
- Best practices for metadata preservation and portability
- Complete extraction scripts with 100% verified recovery rate
- Export tools for popular photo management software (Lightroom, digiKam, Apple Photos)
- Metadata verification and cleaning utilities
Basic Database Parsing
from docs.reference_implementation import IbiDatabaseParser
# Initialize parser
parser = IbiDatabaseParser('path/to/index.db', 'path/to/files/')
parser.connect()
# Get all files with metadata
files = parser.get_all_files()
# Get AI content tags
tags = parser.get_content_tags_summary()
# Verify file recovery rate
recovery = parser.verify_file_recovery_rate()
print(f"Recovery rate: {recovery['recovery_rate']:.1f}%")
File Storage Access
# ibi uses this formula for file storage:
content_id = "jT9JduP8vIHpwuY32gLQ"
file_path = f"/files/{content_id[0]}/{content_id}"
# Results in: /files/j/jT9JduP8vIHpwuY32gLQ
Key Database Queries
-- Get all recoverable files
SELECT f.name, f.contentID, f.mimeType
FROM Files f
WHERE f.contentID IS NOT NULL;
-- Get AI content tags
SELECT f.name, ft.tag
FROM Files f
JOIN FilesTags ft ON f.id = ft.fileID
WHERE ft.auto = 1;
-- Get album organization
SELECT fg.name, f.name
FROM FileGroups fg
JOIN FileGroupFiles fgf ON fg.id = fgf.fileGroupID
JOIN Files f ON fgf.fileID = f.id;
🎁 What Makes This Valuable
For Data Recovery Professionals:
- 100% recovery rate verified on test databases
- 5,312+ AI content tags provide automatic photo categorization
- 50+ user albums with meaningful family names
- Complete EXIF/GPS preservation for geotagged photos
- Standard export formats for popular photo software
For Families:
- Recover not just files, but complete photo organization
- Get searchable content tags (“person”, “child”, “beach”, “graduation”)
- Preserve album structure (“Jon’s graduation”, “Hawaii 2010”)
- GPS locations and camera metadata intact
For Developers:
- Clean, documented schema - no reverse engineering needed
- Reference implementation to get started immediately
- Standard export formats - Lightroom, digiKam, Apple Photos, XMP
- Smart filtering - removes ibi-specific data automatically
📊 Typical Dataset Statistics
- 8,500+ files per device (images, videos, documents)
- ~79% images (JPEG, HEIC, PNG, GIF)
- ~21% videos (MP4, QuickTime)
- 5,300+ AI content tags (computer vision analysis)
- 40-60 user albums with family-meaningful names
- 99-100% file recovery rate (files mapped to physical storage)
✅ Include (Portable Data):
- AI-generated content tags
- User-created album organization
- Original camera EXIF data
- GPS coordinates and location names
- Capture timestamps
❌ Exclude (ibi-Specific Data):
- auth0 user IDs and authentication data
- ibi sharing system IDs (non-functional)
- Internal permission systems
- Custom hash tracking fields
- Device registration data
Adobe Lightroom
Filename,Keywords,Caption,Album,GPS
IMG_001.jpg,"person; child; beach","Family vacation","Hawaii 2010","21.3099,-157.8581"
Name,Tags,Album,Date,Latitude,Longitude
IMG_001.jpg,"People/person|Places/Beach/beach","Hawaii 2010","2018-03-15",21.3099,-157.8581
XMP Sidecar Files
<?xml version="1.0" encoding="UTF-8"?>
<x:xmpmeta xmlns:x="adobe:ns:meta/">
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<rdf:Description xmlns:dc="http://purl.org/dc/elements/1.1/">
<dc:subject>
<rdf:Bag>
<rdf:li>person</rdf:li>
<rdf:li>beach</rdf:li>
</rdf:Bag>
</dc:subject>
</rdf:Description>
</rdf:RDF>
</x:xmpmeta>
🔬 Research Applications
This schema enables research into:
- Photo management system design
- AI content recognition accuracy over time
- User organization patterns in family photo collections
- Mobile device photo storage evolution
- Data preservation best practices
📜 License & Usage
GPLv3 - Use freely with source code sharing:
- ✅ Commercial data recovery services (must open source improvements)
- ✅ Open source recovery tools (contributions must be GPLv3)
- ✅ Academic research
- ✅ Personal recovery projects
- ✅ AI/ML training on photo organization
Source code sharing required for any derivative works or improvements.
Share Your Discoveries
- Schema variations found in different ibi versions
- Additional metadata fields not documented here
- Recovery edge cases and solutions
- Export format improvements
- Web interfaces for non-technical users
- Batch processing for professional recovery services
- Integration with existing photo software
- Mobile apps for on-device recovery
Research Applications
- Machine learning on photo organization patterns
- Digital preservation best practices
- Photo management UX insights
For Data Recovery Professionals:
- Use these tools to offer complete photo recovery services
- 5,312 AI tags make recovery much more valuable than just file extraction
- Album preservation helps families get organized collections back
For Developers:
- docs/reference_implementation.py gets you started immediately
- schema_documentation.md has everything you need to build tools
- api_specification.json enables automated code generation
- GitHub Repository: github.com/alandtse/ibiRecovery
For Researchers:
- Real-world photo management data with user organization patterns
- AI content analysis from actual consumer devices
- Temporal data spanning 2017-2023 timeframe
🎯 Impact
This documentation transforms ibi data recovery from “just getting files back” to “complete photo library reconstruction” with:
- Automatic content categorization (AI tags)
- Preserved family organization (albums)
- Geographic context (GPS data)
- Technical metadata (camera info)
- Universal software compatibility
Result: Families get back not just their photos, but their organized, searchable photo libraries.