thibitisha/kmpdc-seeder

Laravel package to seed KMPDC practitioner data

0.0.47-alpha 2025-11-05 15:32 UTC

Requires

• php: ^8.2
• league/csv: ^9.16
• symfony/css-selector: ^7.0
• symfony/dom-crawler: ^7.0

MIT

• kdbz

README

A Laravel 12+ package for scraping, cleaning, and importing practitioner data from the Kenya Medical Practitioners and Dentists Council (KMPDC) master register.

This package is designed for:

• 🧑‍🏫 Teaching structured data ingestion and seeding in Laravel

🚀 Features

✅ Scrapes KMPDC register HTML into structured CSVs
✅ Extracts & normalizes:

• Practitioner details, qualifications, degrees, institutions
  ✅ Imports structured data into related tables
  ✅ Works seamlessly with Eloquent models

📦 Installation

composer require thibitisha/kmpdc-seeder:^0.0.47-alpha

🧩 Database Schema

The package models relationships between practitioners, qualifications, degrees, institutions, and related entities.

erDiagram

    practitioners ||--|| statuses : has
    practitioners ||--|| specialities : belongs_to
    practitioners }o--|| sub_specialities : may_specialize_in
    practitioners ||--o{ qualifications : holds
    practitioners ||--o{ contacts : has
    practitioners ||--o{ licenses : issued
    practitioners ||--o{ practitioner_documents : uploads

    qualifications }|--|| degrees : is_type
    qualifications }|--|| institutions : awarded_by

    sub_specialities }|--|| specialities : under

    licenses ||--o{ payments : renewed_via

    users ||--|| roles : assigned_role

    verification_logs }o--|| practitioners : attempts_to_verify 

    practitioners {
        BIGINT id PK
        VARCHAR registration_number UK
        VARCHAR full_name
        VARCHAR profile_photo_url "nullable"
        BIGINT status_id FK
        BIGINT speciality_id FK "nullable"
        BIGINT sub_speciality_id FK "nullable"
        DATE date_of_registration "nullable"
        TIMESTAMP created_at
        TIMESTAMP updated_at
    }

    sub_specialities {
        BIGINT id PK
        VARCHAR name
        BIGINT speciality_id FK
    }

Loading

📝 Notes

• All inserts and relationships are handled via Eloquent models

🧩 Model Setup & Relationships

Before running the import commands, ensure that all Eloquent models and their respective relationships are correctly defined according to the database schema provided above.

Each model should reflect its table structure and foreign key relationships.

⚠️ Note: the $fillable propery should also be defined in each model

For example:

• Practitioner

  • belongsTo → Status, Speciality, SubSpeciality
  • hasMany → Contact, License, Qualification, PractitionerDocument, VerificationLog

• Qualification

  • belongsTo → Practitioner, Degree, Institution

• SubSpeciality

  • belongsTo → Speciality

• License

  • belongsTo → Practitioner
  • hasMany → Payment

These relationships are essential for the import command to correctly associate records during data seeding. Ensure that:

• Foreign keys are properly defined in migrations.
• Model relationships use typed relationship methods (as required in Laravel 12).
• Nullable foreign keys (e.g., speciality_id, sub_speciality_id) are handled gracefully to avoid integrity constraint violations.

⚠️ Note: The package assumes these model relationships exist and are correctly implemented. Missing or incorrectly defined relationships may cause the import process to fail or produce inconsistent data.

🔁 Workflow Overview

This package processes practitioner data in three main stages:

Step	Command	Input → Output	Description
🧭 1. Sync	php artisan kmpdc:sync	Web data → timestamped CSV file	Crawls the KMPDC register and generates a timestamped CSV file containing all practitioners' details (name, registration number, qualifications, address, status, speciality, etc.).
🧮 2. Extract	php artisan kmpdc:extract	CSV → JSON files	Parses the latest CSV, extracts structured data (degrees, institutions, statuses, specialities), and saves normalized JSON files.
📥 3. Import	php artisan kmpdc:import	JSON files → Database	Imports structured data into your Laravel models, preserving relationships and handling duplicates safely.

🧩 Run in this order: sync → extract → import

🧭 Step 1 — Sync (Generate CSV)

The sync command crawls the KMPDC register and saves a timestamped CSV file to your Laravel storage directory.

php artisan kmpdc:sync

Output Example:

storage/app/kmpdc-data/csv
└── 0000_00_00_122105_kmpdc_practitioners.csv

Each run produces a uniquely named file based on the timestamp, ensuring previous syncs remain preserved.

Included columns:

• Fullname
• Registration Number
• Address
• Qualifications
• Discipline / Speciality
• Sub-speciality
• Status
• Profile link

🧮 Step 2 — Extract (Generate JSON)

The extract command reads the latest timestamped CSV and produces clean, structured JSON files.

php artisan kmpdc:extract

📥 Step 3 — Import (Save to Database)

Imports the normalized JSON into your relational schema.

php artisan kmpdc:import

🧠 Verify Imports

You can verify successful import via Tinker:

php artisan tinker

use App\Models\Practitioner;

Practitioner::with(['status', 'speciality', 'subSpeciality', 'qualifications'])->first();

Examples

use App\Models\Practitioner;

// Fetch one doctor
$doctor = Practitioner::with(['status', 'speciality', 'subSpeciality', 'qualifications.degree', 'qualifications.institution'])->first();

$doctor->full_name;             // "Dr JOHN DOE"
$doctor->status->name;          // "ACTIVE"
$doctor->speciality?->name;     // "SURGERY"
$doctor->subSpeciality?->name;  // "CARDIOLOGY"

// Get all unique institutions
\App\Models\Institution::pluck('name');

// Find all practitioners under Internal Medicine
\App\Models\Practitioner::whereHas('speciality', fn($q) => $q->where('name', 'INTERNAL MEDICINE'))->count();

🤝 Contributing

1. Fork the repository
2. Create your feature branch (git checkout -b feature/my-feature)
3. Commit your changes (git commit -m 'Add my feature')
4. Push to your branch (git push origin feature/my-feature)
5. Open a Pull Request 🎉

🏛️ Acknowledgement & Disclaimer

This package utilizes publicly accessible practitioner data from the
official Kenya Medical Practitioners and Dentists Council (KMPDC) website.

The author, KDBZ, is not affiliated with, endorsed, or sponsored by KMPDC.
All practitioner data, formats, and associated intellectual property remain the exclusive property of KMPDC.

This project is provided solely for educational, analytical, and research purposes.
It is not intended for redistribution, resale, or use in any commercial or official capacity.

Users are responsible for ensuring that their use of this tool complies with:

• The KMPDC website’s terms of service
• Applicable data protection and privacy laws
• Relevant ethical and professional standards

For any commercial or institutional use, explicit authorization should be sought directly from KMPDC.

📜 License

This package is open-sourced software licensed under the MIT license.

Author: KDBZ Repository: https://github.com/kdbz/kmpdc-seeder