GestionTablets/docs/DATABASE_SCHEMA_SPECIFICATION.md

Database Schema Specification for Student Data Management

Overview

This document specifies the SQLite database schema for managing student data imported from CSV files. The schema is designed to support the tablet lending system while incorporating student information from external data sources.

Source Data Structure

The CSV file Datos_programa.csv has the following structure (ISO-8859-1 encoded):

Nº Or.,"Apellidos, Nombre",Fecha Nac.,C.I.A.L.,NIF/NIE/Pas.,Registro,Expediente,Sexo,Grupo,Estudio

Field Descriptions

Field	Type	Description	Notes
Nº Or.	Integer	Order number	Primary identifier in source
Apellidos, Nombre	String	Full name (Surnames, Name)	Needs splitting into separate fields
Fecha Nac.	Date	Birth date	Format: DD/MM/YYYY
C.I.A.L.	String	Internal identification code	Unique identifier
NIF/NIE/Pas.	String	National ID / Foreign ID / Passport	Unique identifier
Registro	Integer	Registration number	Internal reference
Expediente	String	File number	Optional, may be empty
Sexo	String	Gender	Single character (M/F)
Grupo	String	Study group	Educational level
Estudio	String	Study program	Full description - TO BE DISCARDED

Database Schema

Table: students

The core table for storing student information.

CREATE TABLE IF NOT EXISTS students (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    -- Source identifiers
    order_number INTEGER,
    cial_code TEXT UNIQUE NOT NULL,
    nif_nie_passport TEXT UNIQUE,
    registration_number INTEGER,
    file_number TEXT,
    
    -- Personal information
    first_name TEXT NOT NULL,
    last_name TEXT NOT NULL,
    full_name TEXT NOT NULL,
    
    -- Demographic data
    birth_date TEXT,  -- Format: YYYY-MM-DD
    gender TEXT CHECK(gender IN ('M', 'F', 'O')),
    
    -- Study information
    study_group TEXT,
    
    -- Metadata
    created_at TEXT DEFAULT CURRENT_TIMESTAMP,
    updated_at TEXT DEFAULT CURRENT_TIMESTAMP,
    
    -- Constraints
    CONSTRAINT unique_identification UNIQUE (cial_code, nif_nie_passport)
);

Table: tablets (existing)

CREATE TABLE IF NOT EXISTS tablets (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    brand TEXT NOT NULL,
    model TEXT NOT NULL,
    serial_number TEXT UNIQUE NOT NULL,
    status TEXT DEFAULT 'available' CHECK(status IN ('available', 'loaned', 'maintenance')),
    notes TEXT,
    assigned_to_student INTEGER,
    created_at TEXT DEFAULT CURRENT_TIMESTAMP,
    updated_at TEXT DEFAULT CURRENT_TIMESTAMP,
    FOREIGN KEY (assigned_to_student) REFERENCES students(id)
);

Table: users (existing - for staff)

CREATE TABLE IF NOT EXISTS users (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    name TEXT NOT NULL,
    email TEXT,
    phone TEXT,
    identification TEXT UNIQUE,
    role TEXT DEFAULT 'staff' CHECK(role IN ('admin', 'staff', 'viewer')),
    password_hash TEXT,
    created_at TEXT DEFAULT CURRENT_TIMESTAMP,
    updated_at TEXT DEFAULT CURRENT_TIMESTAMP
);

Table: loans (existing - updated)

CREATE TABLE IF NOT EXISTS loans (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    tablet_id INTEGER NOT NULL,
    student_id INTEGER NOT NULL,
    user_id INTEGER,  -- Staff member who processed the loan
    loan_date TEXT NOT NULL,
    return_date TEXT,
    due_date TEXT,
    status TEXT DEFAULT 'active' CHECK(status IN ('active', 'returned', 'overdue', 'lost')),
    notes TEXT,
    created_at TEXT DEFAULT CURRENT_TIMESTAMP,
    updated_at TEXT DEFAULT CURRENT_TIMESTAMP,
    FOREIGN KEY (tablet_id) REFERENCES tablets(id),
    FOREIGN KEY (student_id) REFERENCES students(id),
    FOREIGN KEY (user_id) REFERENCES users(id)
);

Table: non_loanable_devices (existing)

CREATE TABLE IF NOT EXISTS non_loanable_devices (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    brand TEXT NOT NULL,
    model TEXT NOT NULL,
    serial_number TEXT UNIQUE NOT NULL,
    device_type TEXT NOT NULL,
    location TEXT,
    status TEXT DEFAULT 'available',
    notes TEXT,
    purchase_date TEXT,
    purchase_cost REAL,
    assigned_to_student INTEGER,
    created_at TEXT DEFAULT CURRENT_TIMESTAMP,
    updated_at TEXT DEFAULT CURRENT_TIMESTAMP,
    FOREIGN KEY (assigned_to_student) REFERENCES students(id)
);

Indexes

For optimal query performance, the following indexes are recommended:

-- Student indexes
CREATE INDEX IF NOT EXISTS idx_students_cial ON students(cial_code);
CREATE INDEX IF NOT EXISTS idx_students_nif ON students(nif_nie_passport);
CREATE INDEX IF NOT EXISTS idx_students_name ON students(last_name, first_name);
CREATE INDEX IF NOT EXISTS idx_students_birth_date ON students(birth_date);
CREATE INDEX IF NOT EXISTS idx_students_gender ON students(gender);
CREATE INDEX IF NOT EXISTS idx_students_group ON students(study_group);

-- Loan indexes
CREATE INDEX IF NOT EXISTS idx_loans_tablet ON loans(tablet_id);
CREATE INDEX IF NOT EXISTS idx_loans_student ON loans(student_id);
CREATE INDEX IF NOT EXISTS idx_loans_status ON loans(status);
CREATE INDEX IF NOT EXISTS idx_loans_date ON loans(loan_date);
CREATE INDEX IF NOT EXISTS idx_loans_due ON loans(due_date);

-- Tablet indexes
CREATE INDEX IF NOT EXISTS idx_tablets_serial ON tablets(serial_number);
CREATE INDEX IF NOT EXISTS idx_tablets_status ON tablets(status);
CREATE INDEX IF NOT EXISTS idx_tablets_brand ON tablets(brand);
CREATE INDEX IF NOT EXISTS idx_tablets_student ON tablets(assigned_to_student);

-- User indexes
CREATE INDEX IF NOT EXISTS idx_users_identification ON users(identification);
CREATE INDEX IF NOT EXISTS idx_users_role ON users(role);

Data Import Specification

CSV Processing Requirements

1. Encoding Conversion: Convert from ISO-8859-1 to UTF-8
2. Field Splitting: Split "Apellidos, Nombre" into separate "last_name" and "first_name" fields
3. Date Format Conversion: Convert "DD/MM/YYYY" to "YYYY-MM-DD"
4. Field Discarding: The "Estudio" field will be discarded as per requirements
5. Empty Field Handling: Empty fields should be stored as NULL

Import Mapping

CSV Field	Database Field	Transformation
Nº Or.	order_number	Integer conversion
Apellidos, Nombre	last_name, first_name	Split on ", "
Apellidos, Nombre	full_name	Keep as-is
Fecha Nac.	birth_date	Date format conversion
C.I.A.L.	cial_code	Direct mapping
NIF/NIE/Pas.	nif_nie_passport	Direct mapping
Registro	registration_number	Integer conversion
Expediente	file_number	Direct mapping (may be empty)
Sexo	gender	Direct mapping
Grupo	study_group	Direct mapping
Estudio	-	DISCARDED

Validation Rules

1. Required Fields: cial_code, first_name, last_name
2. Unique Fields: cial_code, nif_nie_passport (if provided)
3. Date Validation: birth_date must be valid date in YYYY-MM-DD format
4. Gender Validation: Must be 'M', 'F', or 'O'

API Endpoints (New/Updated)

Student Management

GET    /students                    - List all students (with pagination)
GET    /students/<int:id>           - Get student details
GET    /students/search             - Search students
POST   /students                   - Create new student
PUT    /students/<int:id>           - Update student
DELETE /students/<int:id>           - Delete student
POST   /students/import            - Import from CSV

Loan Management (Updated)

GET    /loans                       - List all loans
GET    /loans/active                - List active loans
GET    /loans/student/<int:student_id> - Loans for specific student
POST   /loans                       - Create new loan
PUT    /loans/<int:id>/return       - Return tablet

Frontend Updates

New Templates Required

1. students.html - Student list view
2. student_detail.html - Individual student details
3. add_student.html - Add new student form
4. edit_student.html - Edit student form
5. import_students.html - CSV import interface

Updated Templates

1. loan_tablet.html - Add student selection alongside user selection
2. user_loans.html - Update to show student loans
3. index.html - Add student count and recent students

Migration Path

From Existing Database

1. Create new students table
2. Add assigned_to_student field to tablets and non_loanable_devices
3. Add student_id field to loans (nullable initially)
4. Create indexes
5. Import CSV data using import script

Rollback Plan

1. Backup existing database before migration
2. Create migration scripts that are idempotent
3. Test migration on copy of production data
4. Provide rollback SQL scripts

File Locations

project/
├── database/
│   ├── schema.sql              # Complete SQL schema
│   └── migrations/
│       ├── 001_add_students_table.sql
│       ├── 002_add_student_relationships.sql
│       └── 003_add_indexes.sql
├── scripts/
│   ├── import_students.py      # CSV import script
│   └── migrate_database.py     # Database migration script
├── templates/
│   ├── students/
│   │   ├── list.html
│   │   ├── detail.html
│   │   ├── add.html
│   │   ├── edit.html
│   │   └── import.html
│   └── ... (existing templates)
└── app.py                       # Updated with new routes

Testing Requirements

1. Import Script Tests:

   • Test with valid CSV file
   • Test with empty CSV file
   • Test with malformed CSV file
   • Test with duplicate entries
   • Test encoding conversion
   • Test field splitting

2. Database Tests:

   • Test unique constraints
   • Test foreign key relationships
   • Test indexes performance
   • Test data integrity

3. API Tests:

   • Test all CRUD operations for students
   • Test loan operations with students
   • Test search and filtering
   • Test pagination

Security Considerations

1. Data Validation: All inputs must be validated before database operations
2. SQL Injection: Use parameterized queries exclusively
3. Authentication: Student data should only be accessible to authorized users
4. Audit Trail: Maintain logs of all import operations and data modifications
5. Backup: Regular backups of student data

Performance Considerations

1. Pagination: All list endpoints should support pagination (default: 50 items per page)
2. Caching: Consider caching frequently accessed student data
3. Batch Operations: Import script should use batch inserts for performance
4. Index Maintenance: Regularly update statistics for query planner