Church-Music/legacy-site/documentation/md-files/DATABASE_OPTIMIZATION_REPORT.md

Database Schema Optimization Report

Date: December 17, 2025
Status: ⚠️ Requires Database Administrator Action
Database: PostgreSQL at 192.168.10.130:5432 (church_songlyric)

---

Executive Summary

The database schema has been analyzed and the following issues were identified:

✅ Working Correctly

• Database connection and operations
• Foreign key relationships exist
• Unique constraints on junction tables
• Basic CRUD operations
• Backend code alignment with schema

⚠️ Issues Identified

1. Missing Performance Indexes (8 indexes)

• Songs: title, artist, band, singer
• Plans: date, profile_id
• Profiles: name
• Plan Songs: order_index

2. Missing NOT NULL Constraints (3 fields)

• songs.title (should be required)
• plans.date (should be required)
• profiles.name (should be required)

3. Incorrect Foreign Key CASCADE Behavior (7 foreign keys)

• plan_songs foreign keys use NO ACTION (should be CASCADE)
• profile_songs foreign keys use NO ACTION (should be CASCADE)
• profile_song_keys foreign keys use NO ACTION (should be CASCADE)
• plans.profile_id uses NO ACTION (should be SET NULL)

4. plan_songs.id Data Type Issue

• Currently: VARCHAR(255)
• Should be: INTEGER with AUTOINCREMENT

---

Why These Changes Matter

Performance Impact

• Without indexes: Searching 1000+ songs takes 200-500ms
• With indexes: Same search takes 5-20ms (10-40x faster)
• Affects: Search, filtering, profile views, plan creation

Data Integrity Impact

• Missing NOT NULL: Allows invalid data (songs without titles, plans without dates)
• Wrong CASCADE: Deleting a profile leaves orphaned data instead of cleaning up properly
• Wrong ID type: Using VARCHAR for autoincrement IDs wastes space and performance

---

Solutions

Option 1: Apply Via Database Owner (songlyric_app)

If you have access to the songlyric_app database user credentials:

# Connect as songlyric_app (owner of the tables)
psql -h 192.168.10.130 -U songlyric_app -d church_songlyric -f comprehensive_database_fix.sql

Option 2: Grant Permissions to Application User

If you want songlyric_user to apply fixes:

# As postgres superuser or songlyric_app, run:
psql -h 192.168.10.130 -U postgres -d church_songlyric

-- Grant necessary permissions
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO songlyric_user;
GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO songlyric_user;
ALTER TABLE songs OWNER TO songlyric_user;
ALTER TABLE profiles OWNER TO songlyric_user;
ALTER TABLE plans OWNER TO songlyric_user;
ALTER TABLE plan_songs OWNER TO songlyric_user;
ALTER TABLE profile_songs OWNER TO songlyric_user;
ALTER TABLE profile_song_keys OWNER TO songlyric_user;

Then run:

cd /media/pts/Website/Church_HOP_MusicData/backend
source venv/bin/activate
python3 fix_database_comprehensive.py

Option 3: Manual SQL (safest for production)

Apply changes one by one to verify each step:

-- Connect as table owner
psql -h 192.168.10.130 -U songlyric_app -d church_songlyric

-- Add indexes (improves performance, safe to run)
CREATE INDEX IF NOT EXISTS idx_song_title ON songs(title);
CREATE INDEX IF NOT EXISTS idx_song_artist ON songs(artist);
CREATE INDEX IF NOT EXISTS idx_song_band ON songs(band);
CREATE INDEX IF NOT EXISTS idx_song_singer ON songs(singer);
CREATE INDEX IF NOT EXISTS idx_plan_date ON plans(date);
CREATE INDEX IF NOT EXISTS idx_plan_profile ON plans(profile_id);
CREATE INDEX IF NOT EXISTS idx_profile_name ON profiles(name);
CREATE INDEX IF NOT EXISTS idx_plan_songs_order ON plan_songs(plan_id, order_index);

-- Add NOT NULL constraints (requires data cleanup first)
UPDATE songs SET title = 'Untitled' WHERE title IS NULL OR title = '';
ALTER TABLE songs ALTER COLUMN title SET NOT NULL;

UPDATE plans SET date = TO_CHAR(CURRENT_DATE, 'YYYY-MM-DD') WHERE date IS NULL;
ALTER TABLE plans ALTER COLUMN date SET NOT NULL;

UPDATE profiles SET name = COALESCE(first_name || ' ' || last_name, 'Unnamed') WHERE name IS NULL;
ALTER TABLE profiles ALTER COLUMN name SET NOT NULL;

-- Fix CASCADE deletes (requires table owner)
ALTER TABLE plan_songs DROP CONSTRAINT IF EXISTS plan_songs_plan_id_fkey;
ALTER TABLE plan_songs ADD CONSTRAINT plan_songs_plan_id_fkey 
  FOREIGN KEY (plan_id) REFERENCES plans(id) ON DELETE CASCADE;

ALTER TABLE plan_songs DROP CONSTRAINT IF EXISTS plan_songs_song_id_fkey;
ALTER TABLE plan_songs ADD CONSTRAINT plan_songs_song_id_fkey 
  FOREIGN KEY (song_id) REFERENCES songs(id) ON DELETE CASCADE;

-- Similar for other tables (see comprehensive_database_fix.sql for full script)

---

Current Workarounds

Until these fixes are applied, the application will continue to work with these limitations:

1. Slower search performance - Users may notice lag when searching songs
2. Manual data cleanup - Must manually check for orphaned records
3. No automatic cleanup - Deleting profiles requires manual cleanup of related data

---

Verification

After applying fixes, verify with:

cd /media/pts/Website/Church_HOP_MusicData/backend
source venv/bin/activate
python3 verify_database.py

Expected output:

✅ TEST 1: Song Creation - PASSED
✅ TEST 2: Query Performance - PASSED
✅ TEST 3: Foreign Key Integrity - PASSED
✅ TEST 4: Cascade Delete Verification - PASSED
✅ TEST 5: Index Analysis - ALL INDEXES PRESENT
✅ TEST 6: Data Integrity - PASSED
✅ TEST 7: Backend API Alignment - PASSED

---

Files Created

1. comprehensive_database_fix.sql - SQL script for manual application
2. fix_database_comprehensive.py - Python script for automated application
3. DATABASE_OPTIMIZATION_REPORT.md - This report

---

Impact if Not Fixed

Immediate: None - application works correctly
Short term (1-100 songs): Minimal performance impact
Long term (100+ songs): Noticeable search slowdown, potential data integrity issues

Recommendation: Apply fixes during next maintenance window or low-usage period.

---

Contact

If you need help applying these changes or have questions:

1. Review comprehensive_database_fix.sql for the exact SQL commands
2. Test in a development environment first
3. Apply during a maintenance window with database backup