diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 0000000..a5793cd --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,288 @@ +# JIRA AI Fixer - Architecture Document + +## System Overview + +JIRA AI Fixer is a microservice that provides AI-powered issue analysis and automated fix generation for enterprise issue tracking systems. + +## High-Level Architecture + +``` +┌─────────────────────────────────────────────────────────────────────────────────┐ +│ External Systems │ +├─────────────────────────────────────────────────────────────────────────────────┤ +│ │ +│ ┌─────────┐ ┌─────────┐ ┌───────────┐ ┌─────────┐ ┌───────────┐ ┌─────────┐ │ +│ │ JIRA │ │ServiceNow│ │ Zendesk │ │Azure DO │ │ GitHub │ │ GitLab │ │ +│ └────┬────┘ └────┬────┘ └─────┬─────┘ └────┬────┘ └─────┬─────┘ └────┬────┘ │ +│ │ │ │ │ │ │ │ +│ └───────────┴────────────┴─────┬──────┴────────────┴────────────┘ │ +│ │ │ +│ HTTPS Webhooks │ +│ │ │ +└──────────────────────────────────────┼──────────────────────────────────────────┘ + │ +┌──────────────────────────────────────┼──────────────────────────────────────────┐ +│ ▼ │ +│ ┌─────────────────────────────────────────────────────────────────────────┐ │ +│ │ JIRA AI Fixer API │ │ +│ │ (FastAPI + Python 3.11) │ │ +│ ├─────────────────────────────────────────────────────────────────────────┤ │ +│ │ │ │ +│ │ ┌──────────────────────────────────────────────────────────────────┐ │ │ +│ │ │ Webhook Layer │ │ │ +│ │ │ /api/webhook/jira /api/webhook/servicenow /api/webhook/... │ │ │ +│ │ └───────────────────────────────┬──────────────────────────────────┘ │ │ +│ │ │ │ │ +│ │ ┌───────────────────────────────▼──────────────────────────────────┐ │ │ +│ │ │ Adapter Layer │ │ │ +│ │ │ normalize_jira() normalize_servicenow() normalize_zendesk() │ │ │ +│ │ └───────────────────────────────┬──────────────────────────────────┘ │ │ +│ │ │ │ │ +│ │ NormalizedIssue │ │ +│ │ │ │ │ +│ │ ┌───────────────────────────────▼──────────────────────────────────┐ │ │ +│ │ │ Core Analysis Engine │ │ │ +│ │ │ save_and_queue_issue() → analyze_issue() (background) │ │ │ +│ │ └───────────────────────────────┬──────────────────────────────────┘ │ │ +│ │ │ │ │ +│ │ ┌───────────────────────┼───────────────────────┐ │ │ +│ │ │ │ │ │ │ +│ │ ▼ ▼ ▼ │ │ +│ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │ +│ │ │ Code Fetcher │ │ LLM Client │ │ PR Creator │ │ │ +│ │ │ (Gitea) │ │ (OpenRouter) │ │ (Gitea) │ │ │ +│ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │ +│ │ │ │ +│ │ ┌──────────────────────────────────────────────────────────────────┐ │ │ +│ │ │ Callback Layer │ │ │ +│ │ │ post_analysis_to_source() - posts back to original system │ │ │ +│ │ └──────────────────────────────────────────────────────────────────┘ │ │ +│ │ │ │ +│ └─────────────────────────────────────────────────────────────────────────┘ │ +│ │ +│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ +│ │ PostgreSQL │ │ Gitea │ │ OpenRouter │ │ +│ │ Database │ │ (Code Host) │ │ (LLM API) │ │ +│ └──────────────┘ └──────────────┘ └──────────────┘ │ +│ │ +│ JIRA AI Fixer Stack │ +└─────────────────────────────────────────────────────────────────────────────────┘ +``` + +## Component Details + +### 1. Webhook Layer + +Receives HTTP POST requests from external systems. Each endpoint is tailored to the specific payload format of the source system. + +**Responsibilities:** +- Receive webhooks +- Basic validation +- Route to appropriate adapter + +### 2. Adapter Layer (Normalizer) + +Transforms vendor-specific payloads into a normalized internal format. + +**NormalizedIssue Schema:** +```python +class NormalizedIssue(BaseModel): + external_id: str # Original ID in source system + external_key: str # Human-readable key (e.g., "JIRA-123") + source: str # Source system identifier + source_url: str # Link back to original issue + title: str # Issue title/summary + description: str # Full description + priority: str # Priority level + labels: List[str] # Tags/labels + callback_url: str # URL to post results back + metadata: Dict # System-specific extra data +``` + +### 3. Core Analysis Engine + +The heart of the system. Runs as a background task. + +**Pipeline:** +1. `fetch_cobol_files()` - Get source code from repositories +2. `build_analysis_prompt()` - Construct LLM prompt +3. `call_llm()` - Send to OpenRouter API +4. `parse_analysis()` - Extract structured data from response +5. `create_fix_branch_and_pr()` - Generate fix PR +6. `post_analysis_to_source()` - Report results + +### 4. Database Layer + +PostgreSQL stores all issues and their analysis results. + +**Tables:** +```sql +issues ( + id SERIAL PRIMARY KEY, + external_id TEXT, + external_key TEXT, + source TEXT, + source_url TEXT, + title TEXT, + description TEXT, + status TEXT, -- pending, analyzed, error + analysis TEXT, + confidence FLOAT, + affected_files TEXT, -- JSON array + suggested_fix TEXT, + pr_url TEXT, + pr_branch TEXT, + callback_url TEXT, + metadata JSONB, + created_at TIMESTAMP, + analyzed_at TIMESTAMP +) + +integrations ( + id SERIAL PRIMARY KEY, + name TEXT UNIQUE, + type TEXT, + config JSONB, + enabled BOOLEAN, + last_event_at TIMESTAMP, + event_count INT +) +``` + +### 5. Git Integration Layer + +Interfaces with Gitea for: +- Fetching source code +- Creating branches +- Committing fixes +- Opening pull requests + +### 6. Callback Layer + +Posts analysis results back to the source system. Handles different API formats: + +| System | Format | +|--------|--------| +| JIRA | REST API v2 | +| ServiceNow | Table API | +| Zendesk | Tickets API | +| Azure DevOps | Work Items API | +| GitHub | Issues API | +| GitLab | Notes API | + +## Data Flow + +``` +1. Webhook Received + └─► POST /api/webhook/{source} + └─► normalize_{source}(payload) + └─► NormalizedIssue + └─► save_to_database() + └─► queue_background_task() + +2. Background Analysis + └─► analyze_issue() + ├─► fetch_cobol_files() ←── Gitea API + ├─► build_analysis_prompt() + ├─► call_llm() ←── OpenRouter API + ├─► parse_analysis() + ├─► create_fix_branch_and_pr() ──► Gitea API + ├─► update_database() + └─► post_analysis_to_source() ──► Source System API + +3. User Views Dashboard + └─► GET /api/issues + └─► query_database() + └─► return JSON +``` + +## Deployment Architecture + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Docker Swarm Cluster │ +├─────────────────────────────────────────────────────────────┤ +│ │ +│ ┌─────────────────┐ ┌─────────────────┐ │ +│ │ Traefik Proxy │◄────────►│ Let's Encrypt │ │ +│ │ (Edge Router) │ │ (TLS Certs) │ │ +│ └────────┬────────┘ └─────────────────┘ │ +│ │ │ +│ │ jira-fixer.startdata.com.br │ +│ │ │ +│ ┌────────▼────────┐ ┌─────────────────┐ │ +│ │ JIRA AI Fixer │ │ PostgreSQL │ │ +│ │ API (8000) │◄────────►│ (internal) │ │ +│ │ Python 3.11 │ │ │ │ +│ └─────────────────┘ └─────────────────┘ │ +│ │ +└─────────────────────────────────────────────────────────────┘ + +External Services: +- Gitea (gitea.startdata.com.br) - Code repository +- OpenRouter (openrouter.ai) - LLM API +``` + +## Security Considerations + +### Network Security +- All external traffic through HTTPS (TLS 1.3) +- Internal services on isolated Docker network +- Database not exposed externally + +### Authentication +- Webhook secrets (optional) for validation +- Gitea token for repository access +- OpenRouter API key for LLM + +### Data Privacy +- Issue descriptions may contain sensitive data +- LLM calls go to external service (OpenRouter) +- Consider self-hosted LLM for sensitive environments + +## Scalability + +### Current Limits +- Single API instance +- ~50 concurrent analyses +- ~1000 issues/day throughput + +### Scaling Options +1. **Horizontal**: Add more API replicas +2. **Queue**: Add Redis for job queue +3. **Database**: PostgreSQL read replicas +4. **LLM**: Multiple OpenRouter API keys + +## Monitoring + +### Health Check +```bash +GET /api/health +→ {"status": "healthy", "service": "jira-ai-fixer", "version": "2.0.0"} +``` + +### Metrics Endpoint +```bash +GET /api/stats +→ { + "total": 150, + "analyzed": 142, + "prs_created": 98, + "avg_confidence": 85, + "by_source": {"jira": 80, "servicenow": 50, "tickethub": 20} + } +``` + +## Future Enhancements + +1. **Multi-language Support**: Java, Python, JavaScript analysis +2. **Custom LLM Models**: Support for local/private models +3. **Repository Indexing**: Full codebase embeddings for better context +4. **Automated Testing**: Run tests on fix branches +5. **Approval Workflow**: Require human approval before PR + +--- + +*Document Version: 2.0* +*Last Updated: February 2026* +*StartData Engineering* diff --git a/docs/DEVELOPER_GUIDE.md b/docs/DEVELOPER_GUIDE.md new file mode 100644 index 0000000..da57924 --- /dev/null +++ b/docs/DEVELOPER_GUIDE.md @@ -0,0 +1,229 @@ +# JIRA AI Fixer - Developer Guide + +## Overview + +JIRA AI Fixer is a universal AI-powered issue analysis engine that integrates with multiple issue tracking systems to automatically analyze support cases and suggest code fixes. + +## Architecture + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ JIRA AI Fixer │ +├─────────────────────────────────────────────────────────────────┤ +│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ +│ │TicketHub │ │ JIRA │ │ServiceNow│ │ Zendesk │ ... │ +│ │ Webhook │ │ Webhook │ │ Webhook │ │ Webhook │ │ +│ └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘ │ +│ │ │ │ │ │ +│ └─────────────┴──────┬──────┴─────────────┘ │ +│ │ │ +│ ┌───────▼───────┐ │ +│ │ Normalizer │ (Adapter Pattern) │ +│ └───────┬───────┘ │ +│ │ │ +│ ┌───────▼───────┐ │ +│ │ Analyzer │ (LLM + Code Analysis) │ +│ └───────┬───────┘ │ +│ │ │ +│ ┌─────────────┼─────────────┐ │ +│ │ │ │ │ +│ ┌──────▼─────┐ ┌─────▼─────┐ ┌────▼────┐ │ +│ │ Database │ │ PR Gen │ │Callback │ │ +│ │ PostgreSQL │ │ Gitea │ │ to Src │ │ +│ └────────────┘ └───────────┘ └─────────┘ │ +└─────────────────────────────────────────────────────────────────┘ +``` + +## Tech Stack + +- **Language**: Python 3.11 +- **Framework**: FastAPI (async) +- **Database**: PostgreSQL 15 +- **LLM**: OpenRouter API (Llama 3.3 70B free tier) +- **Code Hosting**: Gitea (self-hosted) + +## Project Structure + +``` +jira-ai-fixer/ +├── api/ +│ └── main_v3.py # Main application (monolith) +├── docs/ +│ ├── DEVELOPER_GUIDE.md +│ ├── USER_GUIDE.md +│ └── ARCHITECTURE.md +└── README.md +``` + +## Key Components + +### 1. Webhook Adapters + +Each supported system has a dedicated adapter that normalizes payloads: + +```python +def normalize_jira(payload: dict) -> Optional[NormalizedIssue]: + """Normalize JIRA webhook payload""" + issue = payload.get("issue", {}) + fields = issue.get("fields", {}) + return NormalizedIssue( + external_id=str(issue.get("id")), + external_key=issue.get("key"), + source="jira", + title=fields.get("summary"), + description=fields.get("description"), + callback_url=f"{base_url}/rest/api/2/issue/{issue.get('key')}/comment" + ) +``` + +### 2. Analysis Pipeline + +```python +async def analyze_issue(issue_id: int, issue: NormalizedIssue): + # 1. Fetch source code from repositories + cobol_files = await fetch_cobol_files() + + # 2. Build LLM prompt + prompt = build_analysis_prompt(issue, cobol_files) + + # 3. Call LLM API + analysis = await call_llm(prompt) + + # 4. Parse response + result = parse_analysis(analysis) + + # 5. Create fix branch and PR + pr_info = await create_fix_branch_and_pr(issue, result) + + # 6. Post back to source system + await post_analysis_to_source(issue, result, pr_info) +``` + +### 3. Callback System + +Results are posted back to the source system in their native format: + +| System | Method | Format | +|--------|--------|--------| +| TicketHub | POST /tickets/{id}/comments | `{"author": "...", "content": "..."}` | +| JIRA | POST /rest/api/2/issue/{key}/comment | `{"body": "..."}` | +| ServiceNow | PATCH /api/now/table/incident/{sys_id} | `{"work_notes": "..."}` | +| Zendesk | PUT /api/v2/tickets/{id}.json | `{"ticket": {"comment": {...}}}` | +| Azure DevOps | POST /workitems/{id}/comments | `{"text": "..."}` | +| GitHub | POST /repos/.../issues/{n}/comments | `{"body": "..."}` | + +## Adding a New Integration + +1. Create normalizer function: + +```python +def normalize_newsystem(payload: dict) -> Optional[NormalizedIssue]: + # Extract fields from payload + return NormalizedIssue( + external_id=..., + external_key=..., + source="newsystem", + title=..., + description=..., + callback_url=... + ) +``` + +2. Add webhook endpoint: + +```python +@app.post("/api/webhook/newsystem") +async def webhook_newsystem(payload: dict, background_tasks: BackgroundTasks): + issue = normalize_newsystem(payload) + if not issue: + return WebhookResponse(status="ignored", message="Event not handled") + issue_id = await save_and_queue_issue(issue, background_tasks) + return WebhookResponse(status="accepted", issue_id=issue_id) +``` + +3. Add callback format in `post_analysis_to_source()`: + +```python +elif issue.source == "newsystem": + await client.post(issue.callback_url, json={...}) +``` + +## Environment Variables + +| Variable | Description | Default | +|----------|-------------|---------| +| `DATABASE_URL` | PostgreSQL connection string | `postgresql://jira:jira_secret_2026@postgres:5432/jira_fixer` | +| `OPENROUTER_API_KEY` | OpenRouter API key for LLM | (empty = mock mode) | +| `GITEA_URL` | Gitea instance URL | `https://gitea.startdata.com.br` | +| `GITEA_TOKEN` | Gitea API token | (empty) | +| `COBOL_REPO` | Default repository to analyze | `startdata/cobol-sample-app` | + +## API Endpoints + +### Webhooks + +| Endpoint | Description | +|----------|-------------| +| `POST /api/webhook/tickethub` | TicketHub webhooks | +| `POST /api/webhook/jira` | JIRA webhooks | +| `POST /api/webhook/servicenow` | ServiceNow webhooks | +| `POST /api/webhook/zendesk` | Zendesk webhooks | +| `POST /api/webhook/azure-devops` | Azure DevOps webhooks | +| `POST /api/webhook/github` | GitHub Issues webhooks | +| `POST /api/webhook/gitlab` | GitLab Issues webhooks | +| `POST /api/webhook/generic` | Generic webhook format | + +### Management + +| Endpoint | Description | +|----------|-------------| +| `GET /api/health` | Health check | +| `GET /api/issues` | List issues | +| `GET /api/issues/{id}` | Get issue details | +| `GET /api/integrations` | List integrations | +| `GET /api/stats` | Dashboard statistics | + +## Running Locally + +```bash +# Install dependencies +pip install fastapi uvicorn httpx asyncpg pydantic + +# Run with PostgreSQL +export DATABASE_URL="postgresql://user:pass@localhost:5432/jira_fixer" +uvicorn main:app --reload --port 8000 +``` + +## Testing Webhooks + +```bash +# Test TicketHub webhook +curl -X POST http://localhost:8000/api/webhook/tickethub \ + -H "Content-Type: application/json" \ + -d '{ + "event": "ticket.created", + "timestamp": "2026-02-18T18:00:00Z", + "data": { + "id": 1, + "key": "SUPP-1", + "title": "Test issue", + "description": "Test description" + } + }' + +# Test generic webhook +curl -X POST http://localhost:8000/api/webhook/generic \ + -H "Content-Type: application/json" \ + -d '{ + "id": "123", + "key": "CUSTOM-1", + "title": "Custom issue", + "description": "From custom system", + "source": "my-system", + "callback_url": "https://my-system.com/api/issues/123/comments" + }' +``` + +## License + +MIT License - StartData 2026 diff --git a/docs/USER_GUIDE.md b/docs/USER_GUIDE.md new file mode 100644 index 0000000..c994aa0 --- /dev/null +++ b/docs/USER_GUIDE.md @@ -0,0 +1,223 @@ +# JIRA AI Fixer - User Guide + +## What is JIRA AI Fixer? + +JIRA AI Fixer is an AI-powered system that automatically analyzes support tickets from your issue tracking system, identifies the root cause in your codebase, and creates pull requests with suggested fixes. + +## Supported Platforms + +| Platform | Status | Webhook Endpoint | +|----------|--------|------------------| +| TicketHub | ✅ Active | `/api/webhook/tickethub` | +| JIRA | ✅ Ready | `/api/webhook/jira` | +| ServiceNow | ✅ Ready | `/api/webhook/servicenow` | +| Zendesk | ✅ Ready | `/api/webhook/zendesk` | +| Azure DevOps | ✅ Ready | `/api/webhook/azure-devops` | +| GitHub Issues | ✅ Ready | `/api/webhook/github` | +| GitLab Issues | ✅ Ready | `/api/webhook/gitlab` | +| Custom Systems | ✅ Ready | `/api/webhook/generic` | + +## How It Works + +``` +1. Ticket Created → 2. Webhook Sent → 3. AI Analyzes → 4. PR Created → 5. Result Posted + (JIRA) (automatic) (30 seconds) (Gitea) (back to JIRA) +``` + +### Step-by-Step Flow + +1. **Ticket Created**: A support ticket is created in your issue tracker (JIRA, ServiceNow, etc.) + +2. **Webhook Triggered**: Your issue tracker sends a webhook to JIRA AI Fixer + +3. **AI Analysis**: The system: + - Fetches relevant source code from your repositories + - Sends the ticket description + code to an AI model + - Identifies root cause and affected files + - Generates a fix suggestion + +4. **PR Created**: If a fix is found: + - Creates a new branch (`fix/TICKET-123-auto-fix`) + - Applies the code change + - Opens a Pull Request with full explanation + +5. **Result Posted**: A comment is added to your original ticket with: + - Root cause analysis + - Affected files + - Suggested fix + - Link to the Pull Request + - Confidence score + +## Setting Up Webhooks + +### JIRA + +1. Go to **Settings → System → Webhooks** +2. Click **Create a Webhook** +3. Set URL: `https://jira-fixer.startdata.com.br/api/webhook/jira` +4. Events: Select **Issue → created** +5. Save + +### ServiceNow + +1. Go to **System Web Services → Outbound → REST Message** +2. Create new REST Message pointing to: `https://jira-fixer.startdata.com.br/api/webhook/servicenow` +3. Create a Business Rule on Incident table to trigger on Insert + +### Zendesk + +1. Go to **Admin Center → Apps and integrations → Webhooks** +2. Create webhook with endpoint: `https://jira-fixer.startdata.com.br/api/webhook/zendesk` +3. Create Trigger: **When ticket is created → Notify webhook** + +### Azure DevOps + +1. Go to **Project Settings → Service hooks** +2. Create subscription for **Work item created** +3. Set URL: `https://jira-fixer.startdata.com.br/api/webhook/azure-devops` + +### GitHub + +1. Go to **Repository → Settings → Webhooks** +2. Add webhook: `https://jira-fixer.startdata.com.br/api/webhook/github` +3. Select events: **Issues** +4. Content type: `application/json` + +### GitLab + +1. Go to **Settings → Webhooks** +2. URL: `https://jira-fixer.startdata.com.br/api/webhook/gitlab` +3. Trigger: **Issues events** + +### Custom System (Generic) + +Send a POST request with this format: + +```json +{ + "id": "your-ticket-id", + "key": "PROJ-123", + "title": "Issue title", + "description": "Detailed description of the problem", + "source": "your-system-name", + "priority": "high", + "labels": ["bug", "production"], + "callback_url": "https://your-system.com/api/tickets/123/comments" +} +``` + +## Dashboard + +Access the dashboard at: **https://jira-fixer.startdata.com.br** + +### Features + +- **Real-time Statistics**: Total issues, analyzed, PRs created, average confidence +- **Issue List**: View all processed issues with status +- **Issue Detail**: Click any issue to see full analysis, suggested fix, and PR link +- **Filter by Source**: Filter issues by origin system (JIRA, ServiceNow, etc.) +- **Filter by Status**: Filter by pending, analyzed, or error + +## Understanding Results + +### Analysis Comment Format + +When analysis completes, you'll see a comment like this: + +``` +🤖 AI ANALYSIS COMPLETE +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ + +📋 ROOT CAUSE: +The WS-AVAILABLE-BALANCE field is declared as PIC 9(9)V99 which can only +hold values up to 9,999,999.99. Values above this are truncated. + +📁 AFFECTED FILES: AUTH.CBL + +🔧 SUGGESTED FIX: +──────────────────────────────────────── +Change line 15 from: + 05 WS-AVAILABLE-BALANCE PIC 9(9)V99. +To: + 05 WS-AVAILABLE-BALANCE PIC 9(11)V99. +──────────────────────────────────────── + +🔀 PULL REQUEST CREATED: +──────────────────────────────────────── +Branch: fix/supp-1-auto-fix +PR: #5 +URL: https://gitea.startdata.com.br/startdata/cobol-sample-app/pulls/5 +──────────────────────────────────────── + +📊 CONFIDENCE: 92% + +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +Analyzed by JIRA AI Fixer +``` + +### Confidence Score + +| Score | Meaning | +|-------|---------| +| 90-100% | Very likely correct - review and merge | +| 70-89% | Probably correct - review carefully | +| 50-69% | Uncertain - manual investigation recommended | +| <50% | Low confidence - use as a starting point only | + +## Best Practices + +### Writing Good Ticket Descriptions + +The AI works best with detailed descriptions: + +**Good:** +``` +Transaction auth failing for amounts over $10 million. +- Error: "Insufficient funds" even when balance is adequate +- Affected accounts: Corporate accounts with high limits +- Started after last month's release +- Error code: AUTH-5012 +``` + +**Poor:** +``` +auth broken +``` + +### Reviewing PRs + +1. Always review AI-generated PRs before merging +2. Run your test suite on the fix branch +3. Check if the analysis matches your understanding +4. Look for edge cases the AI might have missed + +## Troubleshooting + +### Issue Not Analyzed + +1. Check webhook delivery in your issue tracker +2. Verify the endpoint URL is correct +3. Check the JIRA AI Fixer dashboard for errors + +### Low Confidence Scores + +1. Provide more detail in ticket descriptions +2. Ensure relevant code is in indexed repositories +3. Check if the issue type is supported + +### PR Not Created + +1. Repository must be connected to Gitea +2. Code must be in indexed directory +3. Fix must be auto-applicable (simple changes work best) + +## Contact + +- **Dashboard**: https://jira-fixer.startdata.com.br +- **Portal**: https://aifixerportal.startdata.com.br +- **Support**: support@startdata.com.br + +--- + +*JIRA AI Fixer - Intelligent Support Case Resolution* +*Created by StartData*