Massive update #18
Open
aievolutionpl wants to merge 34 commits into
Open
Conversation
…eatures-from-mark-xl
Refine the JARVIS cockpit with new ways to interact and clearer feedback, all built on existing backend APIs (no server changes): - Text command bar: type requests instead of speaking; reuses the existing transcript WebSocket path with a new "text" source. - Keyboard shortcuts (/ focus, M mute, Esc stop, L clear log, , settings, ? help) with an on-screen shortcuts overlay. - Toast notification system replacing the single fading error line; surfaces errors, file-intake success, clipboard, and mute state. - Live state pill showing idle/listening/thinking/speaking/muted. - Activity-log Copy/Clear controls and two new HUD metrics (session tokens and today's cost) sourced from the existing /api/usage endpoint. - README updated with the new interactions and a shortcuts table.
…ization-and-integration
Vertical slice across JARVIS: - skills.py: installable catalog of 100 small-business skills across 15 categories (sales, marketing, finance, HR, support, ops, etc.). Hybrid model — prompt-based by default, with executable handlers for invoice, SOP, and agenda generation that save real artifacts. Enable/disable, search, and goal-based recommendation, persisted in SQLite. Active skills are injected into the system prompt. - onboarding.py: first-run discovery flow with a persistent user profile. While active, the system prompt steers JARVIS to learn who the user is and what they want, then recommend/enable relevant skills via new [ACTION:PROFILE], [ACTION:RECOMMEND_SKILLS], and [ACTION:ONBOARD_DONE]. - mcp_registry.py: catalog of connectable external tool servers (Gmail, Notion, Slack, GitHub, Supabase, Stripe, etc.) over MCP, with connect/ disconnect state and connected-tool awareness in the prompt. - memory.py: recall now combines FTS rank with importance, recency, and usage; remember() de-duplicates near-identical facts; added importance decay for stale memories. - server.py: new [ACTION:RUN_SKILL] plus skills/onboarding/MCP context injection, and REST endpoints for skills, MCP, and onboarding. - Frontend: searchable skills browser with category chips and toggles, an MCP connect/disconnect panel, a 5-step onboarding wizard, and responsive styling for narrow screens. - Tests for the new modules (17), all green alongside the existing suite.
…cution-for-tools
Build a runtime provider router so JARVIS can switch its conversational brain and voice without a restart, closing the "provider execution router" and "ElevenLabs runtime TTS" roadmap items. Providers (new providers/ package, side-effect-free, httpx-based): - LLM router: Anthropic (default), OpenAI, Ollama (local), Google Gemini, DeepSeek, Groq. OpenAI-compatible + Gemini request shapes; identical plain-string return contract so the [ACTION:X] pipeline is untouched. - TTS router: Fish Audio (default) + ElevenLabs (voice only). - Claude stays the orchestrator for actions/intent/summaries; only the conversational brain is swappable. A summarize() helper falls back to the active provider when no Anthropic key is set. Server: - Route generate_response and synthesize_speech through the routers (live env reads -> saved keys/selection take effect without restart). - New endpoints: POST /api/settings/active, /api/settings/test-provider, GET /api/settings/ollama-models. Extend status with llm/tts metadata. - Allowlist the new selector env keys; rebuild Anthropic client on key save. Frontend: - New Settings -> Engine section: pick active brain + model (Ollama models fetched live), Ollama base URL, active voice + ElevenLabs voice id, with per-provider Test buttons. Persists via /api/settings/active. Bug fixes: - fix_self sent raw bytes as audio instead of base64 (would crash send). - extract_action dropped any spoken text placed after an [ACTION:X] tag. - Stale-reply discard after barge-in via request-id echo (reqId). - Audio player no longer wedges the orb when the last buffer fails to decode. - 300ms mic-resume debounce after speaking to avoid self-hearing. - Status endpoint uses is_configured() instead of duplicated placeholder check. Tests: providers adapter request shapes, config readers, and extract_action regression (15 tests, no API keys required).
Embed hero SVG in README and refresh roadmap status
…ckend REST APIs, configure frontend on port 5180
…s-polish JARVIS 2.0 — Add Memory, Workflow tabs, REST APIs & port updates
…er-for-user-customization Rebuild JARVIS Control Center widgets
…-documentation Refine connectors, LLM routing, memory/skills prompts, shortcuts, and docs
…completion - Respect the user's JARVIS_LLM_MODEL_ANTHROPIC override in the voice loop; Haiku remains the low-latency default only when no override is set - Update token pricing to Claude Haiku 4.5 rates - Require the completion marker for dispatched tasks, with an mtime-based stall fallback instead of completing on any 100+ char partial output - Standardize the documented frontend port on 5180 (matches vite.config.ts) - Harden start.sh/start.ps1: python/npm detection, auto-open the HUD - Remove dead frontend/src/orb.backup.ts https://claude.ai/code/session_01VdYVFs5Pn1qYVXE6tXcjE2
- New system_control.py: pure per-platform command builders (mac/windows/linux) plus async executors that never raise — open URL/app/path, set volume, media control, lock screen, clipboard, screenshots, terminal launch - actions.py now routes non-macOS hosts through system_control instead of crashing on a missing osascript; mac-only features report gracefully - New action tags wired through the voice loop: OPEN_APP, OPEN_PATH, SET_VOLUME, MEDIA, LOCK_SCREEN, CLIPBOARD, SCREENSHOT — documented in the system prompt and recorded in the action log - tests/test_system_control.py: 17 desktop-free unit tests https://claude.ai/code/session_01VdYVFs5Pn1qYVXE6tXcjE2
- Real hosted endpoints for GitHub, Stripe, and HubSpot MCP servers - Six new connectors: Linear, Sentry, Asana, Atlassian, Zapier, Canva - Google Workspace servers (Gmail/Drive/Calendar) now flagged needs_config with a hint instead of fake placeholder URLs; a per-connection URL or stdio command saved in Settings overrides the catalog launch - Settings API whitelist now covers MCP auth tokens and weather keys, so connector credentials can finally be saved from the UI - Brave Search key exposed in the integrations catalog; .env.example lists all MCP token placeholders https://claude.ai/code/session_01VdYVFs5Pn1qYVXE6tXcjE2
- 11 new prompt skills: daily-brief, weekly-review, focus-sprint, habit-coach, reading-list, meal-planner, workout-plan, travel-packing, learning-plan, gift-ideas, decision-helper - 3 new utility skills: password-generator, unit-converter, markdown-to-html - 5 new executable handlers producing real artifacts: daily brief pulls open tasks from memory; reading list persists across runs; markdown→HTML emits a previewable dark-theme page; passwords use secrets and are never written to disk; unit converter covers length/mass/temperature/data - daily-brief enabled by default on fresh installs - tests/test_skills_handlers.py: 11 tests with artifacts redirected to tmp https://claude.ai/code/session_01VdYVFs5Pn1qYVXE6tXcjE2
- Generate jarvis.png/jarvis.ico locally (scripts/generate_icon.py, Pillow) and wire the icon into the Windows/Linux desktop shortcut installers - Add favicon.svg, web manifest, and theme-color for browser/PWA identity - Let users paste MCP connector tokens and custom server URLs directly on each connector card in Settings (saved via the whitelisted keys API) - Accessibility: aria-labels on icon-only controls, live-region state pill, visible focus rings - README: desktop control support matrix, MCP connector overview, REST and WebSocket API reference, troubleshooting, port 5180 consistency (EN + PL) - CLAUDE.md: architecture map and handoff checks cover system_control, mcp_registry, and the new test suites https://claude.ai/code/session_01VdYVFs5Pn1qYVXE6tXcjE2
…le utilities New executable handlers with tests: writing analyzer (text-stats), JSON formatter, CSV-to-Markdown table, DST-aware timezone converter with spoken city aliases, date calculator (between/add, weekday counts), and checklist builder. New prompt skills: fact-check, product comparison, news digest, deep-dive research, email reply, difficult conversation prep, negotiation prep, elevator pitch, regex builder, git helper, wellness break, and sleep wind-down. https://claude.ai/code/session_01VdYVFs5Pn1qYVXE6tXcjE2
…rovements-4hrvhf
…project-features
…nosc-asystenta Add personal briefing cockpit
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.