322 lines
11 KiB
Markdown
322 lines
11 KiB
Markdown
# Database Schema Specification for Student Data Management
|
|
|
|
## Overview
|
|
|
|
This document specifies the SQLite database schema for managing student data imported from CSV files. The schema is designed to support the tablet lending system while incorporating student information from external data sources.
|
|
|
|
## Source Data Structure
|
|
|
|
The CSV file `Datos_programa.csv` has the following structure (ISO-8859-1 encoded):
|
|
|
|
```csv
|
|
Nº Or.,"Apellidos, Nombre",Fecha Nac.,C.I.A.L.,NIF/NIE/Pas.,Registro,Expediente,Sexo,Grupo,Estudio
|
|
```
|
|
|
|
### Field Descriptions
|
|
|
|
| Field | Type | Description | Notes |
|
|
|-------|------|-------------|-------|
|
|
| Nº Or. | Integer | Order number | Primary identifier in source |
|
|
| Apellidos, Nombre | String | Full name (Surnames, Name) | Needs splitting into separate fields |
|
|
| Fecha Nac. | Date | Birth date | Format: DD/MM/YYYY |
|
|
| C.I.A.L. | String | Internal identification code | Unique identifier |
|
|
| NIF/NIE/Pas. | String | National ID / Foreign ID / Passport | Unique identifier |
|
|
| Registro | Integer | Registration number | Internal reference |
|
|
| Expediente | String | File number | Optional, may be empty |
|
|
| Sexo | String | Gender | Single character (M/F) |
|
|
| Grupo | String | Study group | Educational level |
|
|
| Estudio | String | Study program | Full description - TO BE DISCARDED |
|
|
|
|
## Database Schema
|
|
|
|
### Table: students
|
|
|
|
The core table for storing student information.
|
|
|
|
```sql
|
|
CREATE TABLE IF NOT EXISTS students (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
-- Source identifiers
|
|
order_number INTEGER,
|
|
cial_code TEXT UNIQUE NOT NULL,
|
|
nif_nie_passport TEXT UNIQUE,
|
|
registration_number INTEGER,
|
|
file_number TEXT,
|
|
|
|
-- Personal information
|
|
first_name TEXT NOT NULL,
|
|
last_name TEXT NOT NULL,
|
|
full_name TEXT NOT NULL,
|
|
|
|
-- Demographic data
|
|
birth_date TEXT, -- Format: YYYY-MM-DD
|
|
gender TEXT CHECK(gender IN ('M', 'F', 'O')),
|
|
|
|
-- Study information
|
|
study_group TEXT,
|
|
|
|
-- Metadata
|
|
created_at TEXT DEFAULT CURRENT_TIMESTAMP,
|
|
updated_at TEXT DEFAULT CURRENT_TIMESTAMP,
|
|
|
|
-- Constraints
|
|
CONSTRAINT unique_identification UNIQUE (cial_code, nif_nie_passport)
|
|
);
|
|
```
|
|
|
|
### Table: tablets (existing)
|
|
|
|
```sql
|
|
CREATE TABLE IF NOT EXISTS tablets (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
brand TEXT NOT NULL,
|
|
model TEXT NOT NULL,
|
|
serial_number TEXT UNIQUE NOT NULL,
|
|
status TEXT DEFAULT 'available' CHECK(status IN ('available', 'loaned', 'maintenance')),
|
|
notes TEXT,
|
|
assigned_to_student INTEGER,
|
|
created_at TEXT DEFAULT CURRENT_TIMESTAMP,
|
|
updated_at TEXT DEFAULT CURRENT_TIMESTAMP,
|
|
FOREIGN KEY (assigned_to_student) REFERENCES students(id)
|
|
);
|
|
```
|
|
|
|
### Table: users (existing - for staff)
|
|
|
|
```sql
|
|
CREATE TABLE IF NOT EXISTS users (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
name TEXT NOT NULL,
|
|
email TEXT,
|
|
phone TEXT,
|
|
identification TEXT UNIQUE,
|
|
role TEXT DEFAULT 'staff' CHECK(role IN ('admin', 'staff', 'viewer')),
|
|
password_hash TEXT,
|
|
created_at TEXT DEFAULT CURRENT_TIMESTAMP,
|
|
updated_at TEXT DEFAULT CURRENT_TIMESTAMP
|
|
);
|
|
```
|
|
|
|
### Table: loans (existing - updated)
|
|
|
|
```sql
|
|
CREATE TABLE IF NOT EXISTS loans (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
tablet_id INTEGER NOT NULL,
|
|
student_id INTEGER NOT NULL,
|
|
user_id INTEGER, -- Staff member who processed the loan
|
|
loan_date TEXT NOT NULL,
|
|
return_date TEXT,
|
|
due_date TEXT,
|
|
status TEXT DEFAULT 'active' CHECK(status IN ('active', 'returned', 'overdue', 'lost')),
|
|
notes TEXT,
|
|
created_at TEXT DEFAULT CURRENT_TIMESTAMP,
|
|
updated_at TEXT DEFAULT CURRENT_TIMESTAMP,
|
|
FOREIGN KEY (tablet_id) REFERENCES tablets(id),
|
|
FOREIGN KEY (student_id) REFERENCES students(id),
|
|
FOREIGN KEY (user_id) REFERENCES users(id)
|
|
);
|
|
```
|
|
|
|
### Table: non_loanable_devices (existing)
|
|
|
|
```sql
|
|
CREATE TABLE IF NOT EXISTS non_loanable_devices (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
brand TEXT NOT NULL,
|
|
model TEXT NOT NULL,
|
|
serial_number TEXT UNIQUE NOT NULL,
|
|
device_type TEXT NOT NULL,
|
|
location TEXT,
|
|
status TEXT DEFAULT 'available',
|
|
notes TEXT,
|
|
purchase_date TEXT,
|
|
purchase_cost REAL,
|
|
assigned_to_student INTEGER,
|
|
created_at TEXT DEFAULT CURRENT_TIMESTAMP,
|
|
updated_at TEXT DEFAULT CURRENT_TIMESTAMP,
|
|
FOREIGN KEY (assigned_to_student) REFERENCES students(id)
|
|
);
|
|
```
|
|
|
|
## Indexes
|
|
|
|
For optimal query performance, the following indexes are recommended:
|
|
|
|
```sql
|
|
-- Student indexes
|
|
CREATE INDEX IF NOT EXISTS idx_students_cial ON students(cial_code);
|
|
CREATE INDEX IF NOT EXISTS idx_students_nif ON students(nif_nie_passport);
|
|
CREATE INDEX IF NOT EXISTS idx_students_name ON students(last_name, first_name);
|
|
CREATE INDEX IF NOT EXISTS idx_students_birth_date ON students(birth_date);
|
|
CREATE INDEX IF NOT EXISTS idx_students_gender ON students(gender);
|
|
CREATE INDEX IF NOT EXISTS idx_students_group ON students(study_group);
|
|
|
|
-- Loan indexes
|
|
CREATE INDEX IF NOT EXISTS idx_loans_tablet ON loans(tablet_id);
|
|
CREATE INDEX IF NOT EXISTS idx_loans_student ON loans(student_id);
|
|
CREATE INDEX IF NOT EXISTS idx_loans_status ON loans(status);
|
|
CREATE INDEX IF NOT EXISTS idx_loans_date ON loans(loan_date);
|
|
CREATE INDEX IF NOT EXISTS idx_loans_due ON loans(due_date);
|
|
|
|
-- Tablet indexes
|
|
CREATE INDEX IF NOT EXISTS idx_tablets_serial ON tablets(serial_number);
|
|
CREATE INDEX IF NOT EXISTS idx_tablets_status ON tablets(status);
|
|
CREATE INDEX IF NOT EXISTS idx_tablets_brand ON tablets(brand);
|
|
CREATE INDEX IF NOT EXISTS idx_tablets_student ON tablets(assigned_to_student);
|
|
|
|
-- User indexes
|
|
CREATE INDEX IF NOT EXISTS idx_users_identification ON users(identification);
|
|
CREATE INDEX IF NOT EXISTS idx_users_role ON users(role);
|
|
```
|
|
|
|
## Data Import Specification
|
|
|
|
### CSV Processing Requirements
|
|
|
|
1. **Encoding Conversion**: Convert from ISO-8859-1 to UTF-8
|
|
2. **Field Splitting**: Split "Apellidos, Nombre" into separate "last_name" and "first_name" fields
|
|
3. **Date Format Conversion**: Convert "DD/MM/YYYY" to "YYYY-MM-DD"
|
|
4. **Field Discarding**: The "Estudio" field will be discarded as per requirements
|
|
5. **Empty Field Handling**: Empty fields should be stored as NULL
|
|
|
|
### Import Mapping
|
|
|
|
| CSV Field | Database Field | Transformation |
|
|
|-----------|----------------|----------------|
|
|
| Nº Or. | order_number | Integer conversion |
|
|
| Apellidos, Nombre | last_name, first_name | Split on ", " |
|
|
| Apellidos, Nombre | full_name | Keep as-is |
|
|
| Fecha Nac. | birth_date | Date format conversion |
|
|
| C.I.A.L. | cial_code | Direct mapping |
|
|
| NIF/NIE/Pas. | nif_nie_passport | Direct mapping |
|
|
| Registro | registration_number | Integer conversion |
|
|
| Expediente | file_number | Direct mapping (may be empty) |
|
|
| Sexo | gender | Direct mapping |
|
|
| Grupo | study_group | Direct mapping |
|
|
| Estudio | - | DISCARDED |
|
|
|
|
### Validation Rules
|
|
|
|
1. **Required Fields**: cial_code, first_name, last_name
|
|
2. **Unique Fields**: cial_code, nif_nie_passport (if provided)
|
|
3. **Date Validation**: birth_date must be valid date in YYYY-MM-DD format
|
|
4. **Gender Validation**: Must be 'M', 'F', or 'O'
|
|
|
|
## API Endpoints (New/Updated)
|
|
|
|
### Student Management
|
|
|
|
```
|
|
GET /students - List all students (with pagination)
|
|
GET /students/<int:id> - Get student details
|
|
GET /students/search - Search students
|
|
POST /students - Create new student
|
|
PUT /students/<int:id> - Update student
|
|
DELETE /students/<int:id> - Delete student
|
|
POST /students/import - Import from CSV
|
|
```
|
|
|
|
### Loan Management (Updated)
|
|
|
|
```
|
|
GET /loans - List all loans
|
|
GET /loans/active - List active loans
|
|
GET /loans/student/<int:student_id> - Loans for specific student
|
|
POST /loans - Create new loan
|
|
PUT /loans/<int:id>/return - Return tablet
|
|
```
|
|
|
|
## Frontend Updates
|
|
|
|
### New Templates Required
|
|
|
|
1. **students.html** - Student list view
|
|
2. **student_detail.html** - Individual student details
|
|
3. **add_student.html** - Add new student form
|
|
4. **edit_student.html** - Edit student form
|
|
5. **import_students.html** - CSV import interface
|
|
|
|
### Updated Templates
|
|
|
|
1. **loan_tablet.html** - Add student selection alongside user selection
|
|
2. **user_loans.html** - Update to show student loans
|
|
3. **index.html** - Add student count and recent students
|
|
|
|
## Migration Path
|
|
|
|
### From Existing Database
|
|
|
|
1. Create new `students` table
|
|
2. Add `assigned_to_student` field to `tablets` and `non_loanable_devices`
|
|
3. Add `student_id` field to `loans` (nullable initially)
|
|
4. Create indexes
|
|
5. Import CSV data using import script
|
|
|
|
### Rollback Plan
|
|
|
|
1. Backup existing database before migration
|
|
2. Create migration scripts that are idempotent
|
|
3. Test migration on copy of production data
|
|
4. Provide rollback SQL scripts
|
|
|
|
## File Locations
|
|
|
|
```
|
|
project/
|
|
├── database/
|
|
│ ├── schema.sql # Complete SQL schema
|
|
│ └── migrations/
|
|
│ ├── 001_add_students_table.sql
|
|
│ ├── 002_add_student_relationships.sql
|
|
│ └── 003_add_indexes.sql
|
|
├── scripts/
|
|
│ ├── import_students.py # CSV import script
|
|
│ └── migrate_database.py # Database migration script
|
|
├── templates/
|
|
│ ├── students/
|
|
│ │ ├── list.html
|
|
│ │ ├── detail.html
|
|
│ │ ├── add.html
|
|
│ │ ├── edit.html
|
|
│ │ └── import.html
|
|
│ └── ... (existing templates)
|
|
└── app.py # Updated with new routes
|
|
```
|
|
|
|
## Testing Requirements
|
|
|
|
1. **Import Script Tests**:
|
|
- Test with valid CSV file
|
|
- Test with empty CSV file
|
|
- Test with malformed CSV file
|
|
- Test with duplicate entries
|
|
- Test encoding conversion
|
|
- Test field splitting
|
|
|
|
2. **Database Tests**:
|
|
- Test unique constraints
|
|
- Test foreign key relationships
|
|
- Test indexes performance
|
|
- Test data integrity
|
|
|
|
3. **API Tests**:
|
|
- Test all CRUD operations for students
|
|
- Test loan operations with students
|
|
- Test search and filtering
|
|
- Test pagination
|
|
|
|
## Security Considerations
|
|
|
|
1. **Data Validation**: All inputs must be validated before database operations
|
|
2. **SQL Injection**: Use parameterized queries exclusively
|
|
3. **Authentication**: Student data should only be accessible to authorized users
|
|
4. **Audit Trail**: Maintain logs of all import operations and data modifications
|
|
5. **Backup**: Regular backups of student data
|
|
|
|
## Performance Considerations
|
|
|
|
1. **Pagination**: All list endpoints should support pagination (default: 50 items per page)
|
|
2. **Caching**: Consider caching frequently accessed student data
|
|
3. **Batch Operations**: Import script should use batch inserts for performance
|
|
4. **Index Maintenance**: Regularly update statistics for query planner
|