upskill-event-manager/docs/FEATURE_AI_EVENT_POPULATION.md

# AI-Assisted Event Submission Feature Specification

## Executive Summary

This document specifies an AI-powered feature to reduce friction for HVAC trainers submitting events to a WordPress calendar system. The solution uses a single Anthropic Claude API call to parse unstructured event information and auto-populate form fields, dramatically simplifying the submission process for non-technical users.

## Project Context

### Current State
- **Platform**: WordPress site hosted on Cloudways
- **Base Plugin**: The Events Calendar (TEC) by Modern Tribe
- **Custom Plugin**: Existing custom plugin built on top of TEC for HVAC trainer event submissions
- **User Base**: HVAC trainers who are often not tech-savvy and have limited time
- **Problem**: High friction in manual form entry leading to incomplete or missing event submissions

### Technical Constraints
- **Hosting**: Cloudways shared hosting environment
  - Limited server-side processing capabilities
  - No ability to install additional server software
  - Standard PHP/MySQL stack
  - HTTPS request capabilities
- **Must maintain compatibility** with The Events Calendar plugin data structures
- **Cannot add additional WordPress plugins** - solution must be implemented within existing custom plugin

## Functional Requirements

### Core Feature: AI-Assisted Event Form Population

#### User Flow
1. User navigates to the create event page
2. User sees an "Auto-populate with AI" button/option
3. Upon clicking, a modal dialog appears with:
   - Clear instructions for the user
   - Single rich text input field
   - Submit button
4. User provides one of three input types:
   - **Option A**: URL to a webpage containing event details
   - **Option B**: Copy/pasted text from email/document
   - **Option C**: Brief description with key details
5. System processes input showing:
   - Loading spinner
   - Status messages during processing
6. System populates form fields automatically
7. System displays confidence/completeness matrix
8. User reviews and adjusts populated fields before final submission

#### Input Acceptance Criteria

**URL Input**
- Accept any valid HTTP/HTTPS URL
- Handle various event page formats (EventBrite, custom sites, social media event pages)
- Extract information even from non-standard page structures

**Pasted Text Input**
- Accept unformatted text from emails, PDFs, documents
- Handle various formatting styles and structures
- Parse information regardless of order or format

**Brief Description Input**
- Accept natural language descriptions
- Extract partial information when complete details aren't provided
- Make intelligent assumptions about missing data

#### Output Requirements

**Required Fields to Extract**
- Event title
- Date and time
- Location/venue
- Description
- Organizer
- Cost/pricing
- Registration URL (if available)

**Data Validation Requirements**
- Check against existing venues to avoid duplicates
- Check against existing organizers to avoid duplicates
- Validate date/time formats
- Ensure required fields are populated

**Confidence Reporting**
Display a matrix showing:
- **Completeness**: Whether each field was found (High/Medium/Low/N/A)
- **Confidence**: How confident the AI is in the extracted data (0.0-1.0)

## Technical Architecture

### Simplified Single-API Approach

#### Architecture Decision
Use a single Anthropic Claude API call with web search capabilities enabled, eliminating the need for:
- Separate web scraping services
- Multiple NLP processing steps
- Complex orchestration logic
- Background job processing
- Additional server infrastructure

#### Key Components

1. **Frontend (JavaScript)**
   - Modal interface for input collection
   - AJAX communication with WordPress backend
   - Form field population logic
   - Confidence matrix display

2. **Backend (PHP/WordPress)**
   - Single AJAX endpoint for AI processing
   - API key management
   - Request/response handling
   - Data sanitization and validation

3. **External Service**
   - Anthropic Claude API (claude-3-5-sonnet-20241022 or latest)
   - Web search/browsing capabilities enabled

### Implementation Patterns

#### Pattern 1: Stateless Request Processing
- **Do**: Make each request completely self-contained
- **Do**: Include all context needed in single prompt
- **Do**: Return complete results in one response
- **Don't**: Maintain conversation state between requests
- **Don't**: Make multiple API calls for single submission

#### Pattern 2: Defensive Data Handling
```php
// Always validate and sanitize
$user_input = sanitize_textarea_field($_POST['input'] ?? '');
if (empty($user_input)) {
    wp_send_json_error(['message' => 'No input provided']);
    wp_die();
}
```

#### Pattern 3: Structured Output Enforcement
- **Do**: Request JSON output explicitly in prompt
- **Do**: Provide exact schema in prompt
- **Do**: Include example of expected output
- **Don't**: Parse free-text responses
- **Don't**: Rely on consistent formatting without schema

#### Pattern 4: Graceful Degradation
- **Do**: Provide fallback for API failures
- **Do**: Allow manual form filling if AI fails
- **Do**: Cache successful extractions
- **Don't**: Block user progress on AI failure
- **Don't**: Require AI for form submission

### Anti-Patterns to Avoid

#### Anti-Pattern 1: Client-Side API Keys
**Never** expose API keys in JavaScript or make direct API calls from browser

#### Anti-Pattern 2: Synchronous Long-Running Requests
**Avoid** blocking UI during API calls; always use async patterns with loading states

#### Anti-Pattern 3: Over-Engineering
**Don't** build complex parsing systems when Claude can handle it in one call

#### Anti-Pattern 4: Rigid Field Matching
**Don't** require exact field matches; use fuzzy matching for venues/organizers

## Prompt Engineering Strategy

### Core Prompt Structure

```text
You are an AI assistant specialized in extracting event information from various sources.
Your task is to extract structured event data for an HVAC training calendar.

[CONTEXT]
- These are professional training events for HVAC technicians
- Events may be in-person or virtual
- Common organizers include: {list_of_existing_organizers}
- Common venues include: {list_of_existing_venues}

[INPUT]
{user_provides_one_of_these}
- URL: Please retrieve and analyze the webpage at: {url}
- Text: Parse the following event information: {pasted_text}
- Description: Extract event details from: {brief_description}

[EXTRACTION RULES]
1. Match venues/organizers to existing ones when similarity > 80%
2. Infer missing information when reasonable (mark as low confidence)
3. Handle relative dates (convert "next Tuesday" to actual date)
4. Extract all URLs found in source material
5. Standardize time to 24-hour format
6. If multiple events found, extract only the first/primary one

[OUTPUT SCHEMA]
Return ONLY a valid JSON object with this exact structure:
{json_schema_here}

[IMPORTANT]
- Return ONLY the JSON object, no explanatory text
- Use null for missing fields rather than empty strings
- Include confidence scores for each field
```

### Prompt Optimization Techniques

#### 1. Few-Shot Examples
Include 2-3 examples of successful extractions in the system prompt to improve consistency.

#### 2. Role Definition
Clearly define Claude's role as an "event information extraction specialist" to improve focus.

#### 3. Explicit Constraints
- Specify exact date format (ISO 8601)
- Define confidence score ranges (0.0-1.0)
- List acceptable values for enums

#### 4. Context Injection
Always include:
- Current date/time for relative date resolution
- List of existing venues/organizers
- Common patterns in your specific domain

### Handling Edge Cases

#### Edge Case 1: Multiple Events
**Strategy**: Extract only the first/most prominent event
**Prompt Addition**: "If multiple events are found, extract only the primary/first event"

#### Edge Case 2: Ambiguous Dates
**Strategy**: Request current date context
**Implementation**: Always include `Today is {current_date}` in prompt

#### Edge Case 3: Missing Critical Information
**Strategy**: Return partial data with low confidence
**Validation**: Check for minimum required fields before population

## Input/Output Validation

### Input Validation Strategy

#### Pre-Processing Validation
```javascript
function validateInput(input) {
    // Check for minimum length
    if (input.length < 10) {
        return { valid: false, error: 'Input too short' };
    }
    
    // Detect input type
    const type = detectInputType(input);
    
    // URL validation
    if (type === 'url') {
        try {
            new URL(input);
        } catch {
            return { valid: false, error: 'Invalid URL format' };
        }
    }
    
    // Size limits (prevent token overflow)
    if (input.length > 50000) {
        return { valid: false, error: 'Input too large' };
    }
    
    return { valid: true, type };
}
```

#### Input Sanitization
- Strip potentially harmful HTML/scripts
- Normalize whitespace and line breaks
- Remove non-printable characters
- Truncate to maximum acceptable length

### Output Validation Strategy

#### Schema Validation
```php
function validate_ai_response($response) {
    $required_fields = ['title', 'date'];
    $schema = [
        'title' => 'string',
        'date' => 'date',
        'venue' => 'string|null',
        'confidence' => 'array',
        'completeness' => 'array'
    ];
    
    // Check JSON structure
    if (!is_array($response)) {
        throw new ValidationException('Invalid JSON response');
    }
    
    // Validate required fields
    foreach ($required_fields as $field) {
        if (empty($response[$field])) {
            throw new ValidationException("Missing required field: {$field}");
        }
    }
    
    // Type checking and sanitization
    // ... implementation details
    
    return $sanitized_response;
}
```

#### Confidence Threshold Handling
- **High Confidence (>0.8)**: Auto-populate without warning
- **Medium Confidence (0.5-0.8)**: Populate with visual indicator
- **Low Confidence (<0.5)**: Populate but highlight for review
- **No Confidence (null)**: Leave field empty

#### Duplicate Detection
```php
function check_duplicate_event($event_data) {
    // Check for same date + similar title
    $similar_events = get_posts([
        'post_type' => 'tribe_events',
        'meta_key' => '_EventStartDate',
        'meta_value' => $event_data['date'],
        'meta_compare' => '='
    ]);
    
    foreach ($similar_events as $existing) {
        $similarity = similar_text(
            strtolower($existing->post_title),
            strtolower($event_data['title']),
            $percent
        );
        
        if ($percent > 75) {
            return [
                'is_duplicate' => true,
                'existing_id' => $existing->ID,
                'similarity' => $percent
            ];
        }
    }
    
    return ['is_duplicate' => false];
}
```

## Error Handling and Recovery

### API Error Handling

#### Rate Limiting
- Implement exponential backoff
- Cache successful responses for 24 hours
- Show user-friendly message when limit reached

#### Network Failures
- Timeout after 30 seconds
- Provide option to retry
- Fall back to manual entry

#### Invalid Responses
- Log malformed responses for debugging
- Show generic error to user
- Provide option to try different input

### User Communication

#### Status Messages
1. "Analyzing your input..." (0-2 seconds)
2. "Extracting event information..." (2-10 seconds)
3. "Validating extracted data..." (10-15 seconds)
4. "Populating form fields..." (15+ seconds)

#### Error Messages
- Be specific but not technical
- Provide actionable next steps
- Never expose internal errors or API keys

## Security Considerations

### API Key Management
- Store in wp-config.php or environment variables
- Never commit to version control
- Rotate keys periodically
- Use WordPress nonce for AJAX calls

### Input Security
- Sanitize all user inputs
- Prevent SQL injection via parameterized queries
- Escape output when displaying
- Limit request frequency per user

### Data Privacy
- Don't log sensitive event information
- Clear temporary data after processing
- Comply with GDPR/privacy requirements
- Allow users to opt-out of AI features

## Performance Optimization

### Caching Strategy
- Cache AI responses for identical inputs (24-hour TTL)
- Cache venue/organizer lists (1-hour TTL)
- Use WordPress transients for temporary data

### Request Optimization
- Minimize prompt size while maintaining clarity
- Pre-process input to remove redundant information
- Batch similar requests when possible

### Frontend Optimization
- Lazy-load AI modal components
- Debounce input validation
- Use optimistic UI updates

## Testing Strategy

### Test Scenarios
1. **URL Input**: Test with EventBrite, Facebook Events, custom sites
2. **Text Input**: Test with various email formats, PDF copies
3. **Description Input**: Test with minimal and detailed descriptions
4. **Edge Cases**: Test with non-English content, past events, recurring events
5. **Failure Cases**: Test with invalid URLs, gibberish input, timeout scenarios

### Validation Testing
- Ensure all required fields are validated
- Test confidence score accuracy
- Verify duplicate detection works correctly
- Test venue/organizer matching logic

## Implementation Checklist

### Phase 1: Core Infrastructure
- [ ] Set up Anthropic API integration
- [ ] Create WordPress AJAX endpoint
- [ ] Implement basic input validation
- [ ] Build JSON response parser

### Phase 2: UI Implementation
- [ ] Create modal interface
- [ ] Implement form field population
- [ ] Add loading states and progress indicators
- [ ] Build confidence matrix display

### Phase 3: Data Processing
- [ ] Implement venue/organizer matching
- [ ] Add duplicate detection
- [ ] Create validation pipeline
- [ ] Add error handling and recovery

### Phase 4: Optimization
- [ ] Implement caching layer
- [ ] Add request rate limiting
- [ ] Optimize prompt for accuracy
- [ ] Performance testing and tuning

### Phase 5: Polish and Testing
- [ ] Comprehensive error messages
- [ ] User documentation
- [ ] Edge case handling
- [ ] Security audit

## Success Metrics

### Technical Metrics
- API response time < 10 seconds (95th percentile)
- Extraction accuracy > 85% for standard inputs
- Zero security vulnerabilities
- 99.9% uptime for feature availability

### User Metrics
- 50% reduction in time to submit events
- 75% of submissions use AI assistance
- < 5% error rate requiring manual intervention
- User satisfaction score > 4/5

## Appendix: The Events Calendar Integration

### Relevant Post Types
- `tribe_events` - Main event post type
- `tribe_venue` - Venue post type
- `tribe_organizer` - Organizer post type

### Key Meta Fields
- `_EventStartDate` - Event start date/time
- `_EventEndDate` - Event end date/time
- `_EventVenueID` - Reference to venue post
- `_EventOrganizerID` - Reference to organizer post
- `_EventCost` - Event cost
- `_EventURL` - Registration/info URL

### Integration Points
- Use TEC's built-in functions when available
- Respect TEC's data validation rules
- Maintain compatibility with TEC updates
- Leverage TEC's duplicate detection if available