Repository navigation
Feature/muse - #4
Merged
Merged
Conversation
forward = Right down = Left
Added new blocks (Move and Color) Only Forward command is working with vex WORK IN PROGRESS
Previous block functionality still working on new block functions
Found problem: the number for the muscle energy window and the raw data doesn't match up
- need to connect the vex through AP before starting the application
This reverts commit ceb24df.
- fixed colors
…into feature/ganglion
- no longer degrees for turn block (for kids) - Shortcut is ready to be used (Neuroscope-EMG) -Updated gitignore (pycache)
* src/main/index.js: - Fixed production build path from loadURL to loadFile with correct relative path - Added Python process spawning with virtual environment detection - Implemented WebSocket connection validation with 10s timeout - Added proper process cleanup on app quit and window close - Updated hardcoded references from Muse to Ganglion devices - Fixed port configuration from 3000 to 3002 * src/renderer/js/muse-client.js: - Fixed duplicate filters property that was overriding device name filters - Added Ganglion service (0xfe84) to optionalServices for proper discovery * package.json: - Updated electron-builder config to include build/renderer directory - Added extraResources for Python executable bundling - Enhanced build scripts with python:build for PyInstaller integration - Updated serve script to use port 3002 - Added build:local script for complete build process * VEXServer.spec (new): - PyInstaller specification for Python server executable - Proper dependency handling for websockets and asyncio - Console executable configuration for debugging * resources/python/VEXServer_dev.py (new): - Created simulation mode WebSocket server for development - Mock VEX robot with full command protocol (led_on, move, turn_left, turn_right) - Enables testing without physical VEX robot hardware - Runs on ws://127.0.0.1:8777 with JSON command interface * requirements.txt: - Added PyInstaller>=5.0 for standalone executable creation **Key Improvements:** - Enables successful Windows executable creation with integrated Python server - Two-process architecture provides reliable communication in both dev and production - Fixed Bluetooth device scanning to properly detect Ganglion devices instead of Muse - Automatic virtual environment detection in development mode - Production builds now bundle Python server as standalone executable - Enhanced development workflow with simulation server for hardware-free testing
…rking WebSocket management **New Features:** * Console UI: Replace frequency bands display with clean console output * VEX Reconnect: Add functional reconnect button to restart VEX connection without app restart * Enhanced Testing: Complete VEX simulation server with comprehensive test suite * Robust WebSocket Management: Proper connection handling with reconnection support **Console System (NEW):** * src/renderer/js/console.js: Clean terminal-like console without timestamps/emojis * src/renderer/index.html: Replace bands display with console container * src/renderer/js/main.js: Integrate Console class replacing BandPowerVis * src/renderer/js/wrapper-functions.js: Connect blockly_print to console output **VEX Reconnect System (FULLY FUNCTIONAL):** * src/main/index.js: Complete WebSocket connection management with proper scoping - Added stopPythonServer() and reconnectVEX() process management - Moved WebSocket variables and functions to global scope for proper access - Implemented createWebSocketConnection() with timeout and error handling - Fixed function scoping issues that prevented reconnection * src/main/preload.js: Expose vexReconnect IPC method to renderer * src/renderer/index.html: Add VEX reconnect button in sidebar (orange refresh icon) * src/renderer/js/events.js: Implement reconnect button handler with visual feedback **Enhanced VEX Testing:** * resources/python/VEXServer_dev.py: Complete simulation server with full command support * test_vex_server.py: Comprehensive WebSocket test suite for validation **Key Technical Improvements:** - Fixed variable scoping: ws and createWebSocketConnection now accessible globally - Proper WebSocket lifecycle management: Close old connections before creating new ones - Enhanced error handling with timeout protection and graceful fallbacks - Clean console replaces cluttered frequency bands display - Robust reconnection process with proper server restart sequence - Enhanced print blocks accept any data type and output to console **WebSocket Reconnection Flow:** 1. Close existing WebSocket connection 2. Stop Python VEX server gracefully (SIGTERM with SIGKILL fallback) 3. Wait 2 seconds for cleanup 4. Restart Python VEX server 5. Wait 3 seconds for server initialization 6. Re-establish WebSocket connection with timeout protection 7. Resume VEX command functionality
**Block-to-Text Converter:** * src/renderer/js/simple-text-view.js: Text coding environment using Blockly's built-in code generation * src/renderer/index.html: Add toggle buttons for block/text modes and export functionality * src/renderer/js/main.js: Integrate text view with existing Blockly system
… app rename Python (resources/python): Add resilient AIM connection loop; server stays alive and retries when robot is offline Add reconnect_robot action to reset robot connection without restarting server Add disconnect_robot cleanup to close websocket threads and sockets before re-init Add status action reporting robot_connected for UI Improve connection logs (attempting / connected / retry) Electron main (src/main): Detect and spawn bundled VEXServer.exe in production Replace waitOn websocket probe with lightweight TCP port polling Guard reconnectVEX against overlap and clear force-kill timer to avoid killing new process Periodically poll server status and relay to renderer via vex-status Fix false-positive connected state by distinguishing wsConnected vs robotConnected Add lazy-init UDP sockets for Tello (no bind to 9000/8890 until needed) Renderer: Add connectivity status dot showing wsConnected (grey/yellow) and robotConnected (green) Add onVexStatus and requestVexStatus to preload for UI updates Packaging/build: Ensure PyInstaller bundle includes vex module/resources and fix spec path resolution Use process.resourcesPath/python/VEXServer.exe in production and improve spawn/path logging Naming/branding: Rename app to "NeuroBlock EMG for VEX" (productName/appId, window title, HTML title) Docs: Update README with "Adding a New Blockly Block (Example: VEX Kicker)" section and build notes Notes: Reconnect button sends reconnect_robot when local websocket is up; only restarts Python process if websocket is down Status dot: grey = websocket down, yellow = websocket up/robot down, green = robot connected Avoid binding to port 9000 unless Tello is actively used
This reverts commit b031569.
…X integration ### Application Branding & Identity - Rename app from 'NeuroBlock EMG for VEX' → 'NeuroBlock EEG for VEX' - Update package.json productName, window title, and HTML title - Update appId to reflect EEG focus: com.htil.neuroblock-emg-vex ### EEG Signal Processing (Muse Headset) - Restore Signal class for real-time EEG data buffering (512-sample buffer) - Restore BandPowerVis for frequency band visualization (delta, theta, alpha, beta, gamma) - Restore FeatureExtractor for band power computation from raw EEG samples - Restore BLE (Bluetooth Low Energy) integration for Muse headset connection - Wire Muse eegReadings callback → Signal.add_data() for live data streaming ### UI Layout - Two-Panel Design - Left panel (45% width): * Top card (40vh): 4 EEG channel graphs (channels 0-3 from Muse) * Bottom card (40vh): Band power frequency spectrum visualization - Right panel (55% width): * Blockly visual programming editor for robot control * VEX AIM robot command center and status display * Console output and device connection status
…band power metrics and fixing controls_if handler
There was a problem hiding this comment.
Pull request overview
This PR moves Muse + VEX support into the main repo by adding a Python WebSocket VEX server (plus dev mock), Electron main-process wiring/IPC bridges for robot commands + reconnect/status, and new/updated Blockly + UI pieces for VEX control and block-to-text/text-mode features.
Changes:
- Add Python VEX WebSocket server (prod + dev mock) and VEX AIM API modules/resources; wire Electron main process to spawn/communicate with the server via WebSocket and expose IPC APIs to the renderer.
- Extend Blockly with VEX blocks (movement + kicker) and add new “muscle energy” support; update wrapper/interpreter mappings.
- Add/adjust UI + tooling for text-mode/block-to-text experiments and code export; update packaging/build scripts for production distribution.
Reviewed changes
Copilot reviewed 38 out of 51 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| test_vex_server.py | Standalone Python script to manually test VEX WebSocket server commands. |
| test-reconnect.js | Node script to validate WebSocket reconnect behavior. |
| test-converter.html | Browser test harness for BlockToTextConverter. |
| src/renderer/js/wrapper-functions.js | Adds safer band power getters and new VEX command wrappers (incl. kicker). |
| src/renderer/js/text-editor.js | Adds Monaco-based text editor component (CDN-loaded). |
| src/renderer/js/simple-text-view.js | Adds a simple blocks→text display mode and export UI behavior. |
| src/renderer/js/muse-client.js | Updates Web Bluetooth device request options (optional services). |
| src/renderer/js/main.js | Initializes app components; adds band power sanitization and exposes sendCommand. |
| src/renderer/js/interpreter-api.js | Exposes new VEX/kicker functions and muscle energy accessor to js-interpreter. |
| src/renderer/js/feature-extractor.js | Makes relative band power computation safe for missing channel/zero denominator. |
| src/renderer/js/events.js | Adds exportCode and vex-reconnect UI events and corresponding handlers. |
| src/renderer/js/customblock.js | Adds/changes Blockly blocks (muscle_energy, VEX blocks, etc.). |
| src/renderer/js/console.js | Adds a custom in-app console UI component for output/logging. |
| src/renderer/js/coding-mode-manager.js | Adds a block/text coding mode manager using Monaco + Blockly generator. |
| src/renderer/js/categories.js | Adds muscle_energy to Data and introduces a new VEX toolbox category. |
| src/renderer/js/blockly-main.js | Adds VEX category to toolbox; includes muscle_energy in DATA callback. |
| src/renderer/js/block-to-text-converter.js | Adds a converter to generate Python-like text from Blockly workspaces. |
| src/renderer/index.html | Updates UI layout; adds mode toggle controls, export button, VEX reconnect/status UI. |
| src/main/tello.js | Changes Tello initialization to lazy-create the instance on demand. |
| src/main/preload.js | Exposes VEX IPC bridges, status callbacks, and sendCommand to renderer. |
| src/main/index.js | Spawns/stops Python server, maintains WS to VEX server, routes IPC commands and status polling. |
| src/main/index copy.js | Minor formatting change in dev reload try/catch. |
| setup-venv.sh | Adds a helper script to create/activate a Python venv and install deps. |
| resources/python/vex/vex_types.py | Adds VEX AIM API type definitions and globals/constants. |
| resources/python/vex/vex_messages.py | Adds message object definitions for AIM WebSocket commands. |
| resources/python/vex/vex_globals.py | Adds VEXcode-style global aliases/constants. |
| resources/python/vex/settings.py | Adds settings loader for JSON configuration. |
| resources/python/vex/settings.json | Adds VEX connection host configuration. |
| resources/python/vex/pycache/vex_types.cpython-313.pyc | Compiled Python artifact added (should not be committed). |
| resources/python/vex/pycache/vex_globals.cpython-313.pyc | Compiled Python artifact added (should not be committed). |
| resources/python/vex/pycache/settings.cpython-313.pyc | Compiled Python artifact added (should not be committed). |
| resources/python/vex/pycache/init.cpython-313.pyc | Compiled Python artifact added (should not be committed). |
| resources/python/vex/init.py | Exposes VEX AIM API package modules for server use. |
| resources/python/VexTest.py | Test client script for VEX WebSocket server. |
| resources/python/VEXServer_dev.py | Mock VEX server for development without hardware. |
| resources/python/VEXServer.py | Production VEX server with reconnect/status/kicker support and deferred robot init. |
| requirements.txt | Adds Python dependencies (websockets, pyinstaller, etc.). |
| programs/maze_program_emg.xml | Adds an example Blockly program using muscle_energy + VEX actions. |
| package.json | Updates ports/build metadata; adds ws/wait-on deps; adds Python build + extraResources packaging. |
| block-text-demo.html | Adds a standalone demo page for block-to-text conversion. |
| VEXServer.spec | Adds PyInstaller spec to bundle VEXServer and resources. |
| README.md | Replaces placeholder with full project + build docs and a Blockly→IPC→Python→robot flow example. |
| Neuroscope.bat | Adds a Windows helper script to run the app via venv + yarn serve. |
| Neuroscope-EMG.lnk | Adds a Windows shortcut file (binary artifact; should not be committed). |
| .gitignore | Adds ignores for pyc/venv/build and vex pycache. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| import { javascriptGenerator, Order } from "blockly/javascript"; | ||
|
|
||
| let drone_blocks_color = 70; | ||
| let dr // 1. Define the block's JSONks_color = 70; |
Comment on lines
+153
to
+160
| import('./coding-mode-manager.js').then(({ codingModeManager }) => { | ||
| if (codingModeManager && codingModeManager.isInitialized) { | ||
| codingModeManager.exportCode('py'); | ||
| } else { | ||
| console.warn('Coding mode manager not initialized. Using fallback export.'); | ||
| // Fallback: export Blockly-generated JavaScript | ||
| const code = this.blockly.getLatestCode(); | ||
| const blob = new Blob([code], { type: 'text/plain' }); |
Comment on lines
+114
to
+117
| // Resize Blockly | ||
| if (this.blocklyWorkspace) { | ||
| Blockly.svgResize(this.blocklyWorkspace); | ||
| } |
Comment on lines
+78
to
+88
| <div style="position: absolute; right: 5px; top: 260px; z-index: 100"> | ||
| <button id="exportCode" class="huge circular ui teal icon button" style="display: inherit; margin-left: auto" | ||
| title="Export code as Python file"> | ||
| <i class="download icon"></i> | ||
| </button> | ||
| </div> | ||
|
|
||
| <div style="position: absolute; right: 5px; top: 260px; z-index: 100"> | ||
| <button id="bluetooth" class="huge circular ui blue icon button" style="display: inherit; margin-left: auto"> | ||
| <i class="bluetooth icon"></i> | ||
| </button> |
| '''A percentage unit that represents a value from 0% to 100%''' | ||
| LEFT = TurnType.LEFT | ||
| '''A turn unit that is defined as left turning.''' | ||
| RIGHT = TurnType.LEFT |
| "vue:serve": "parcel ./src/renderer/index.html -p 3000 --dist-dir build/renderer", | ||
| "vue:serve": "parcel ./src/renderer/index.html -p 3005 --dist-dir build/renderer", | ||
| "vue:build": "cross-env NODE_ENV=production parcel build ./src/renderer/index.html --dist-dir build/renderer --public-url ./ --no-source-maps --no-scope-hoist", | ||
| "python:build": "C:/dev/repos/VexEMG/neuroscope/.venv/Scripts/pyinstaller.exe VEXServer.spec --clean --noconfirm", |
| const fs = require("fs"); | ||
| const tello = require("./tello.js"); | ||
| const WebSocket = require('ws'); | ||
| const waitOn = require('wait-on'); |
Comment on lines
377
to
+386
| // Emitted when the window is closed. | ||
| win.on("closed", () => { | ||
| // Dereference the window object, usually you would store windows | ||
| // in an array if your app supports multi windows, this is the time | ||
| // when you should delete the corresponding element. | ||
| // Clear the status polling interval | ||
| clearInterval(statusInterval); | ||
|
|
||
| // Close WebSocket if open | ||
| if (ws) { | ||
| try { ws.close(); } catch { } | ||
| ws = null; | ||
| } |
Comment on lines
15
to
+66
| @@ -24,19 +63,32 @@ export const NeuroScope = class { | |||
| this.ble = new BLE(this.signal_handler.add_data.bind(this.signal_handler)); | |||
| this.feature_extractor = new FeatureExtractor(256); | |||
| this.blocklyMain.start(); | |||
| setTimeout(() => { | |||
| //this.ble.build_ble_modal_list(["device1", "device2"]); | |||
| }, 3000); | |||
|
|
|||
Comment on lines
+235
to
+253
| showMessage(message, type = 'info') { | ||
| console.log(`[${type.toUpperCase()}] ${message}`); | ||
|
|
||
| // Create toast message | ||
| const toast = document.createElement('div'); | ||
| toast.className = `ui ${type} message`; | ||
| toast.style.cssText = ` | ||
| position: fixed; | ||
| top: 20px; | ||
| right: 20px; | ||
| z-index: 1000; | ||
| max-width: 300px; | ||
| animation: slideIn 0.3s ease; | ||
| `; | ||
| toast.innerHTML = ` | ||
| <i class="close icon" onclick="this.parentElement.remove()"></i> | ||
| <div class="header">${type.charAt(0).toUpperCase() + type.slice(1)}</div> | ||
| <p>${message}</p> | ||
| `; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Moving Muse + Vex to main Repo