IT Administrators 1-DRC INSIGHT - Central Office Services (COS) Overview Audience: IT Administrators / MSP Engineers Purpose: Explain what COS is, what it does, and how it fits into DRC INSIGHT testing. Overview DRC INSIGHT Central Office Services (COS) is a local content hosting system that allows testing devices to retrieve assessment content from a local server instead of relying entirely on internet delivery. Key Concept: COS is a service-based HTTP system, not a file share. Testing devices do not browse or mount /opt/CentralOffice . How COS Fits into DRC INSIGHT DRC’s training materials describe COS as the content hosting layer for testing devices. When COS is available, test content is delivered locally. If COS is unavailable, content can fall back to DRC cloud delivery. :contentReference[oaicite:1]{index=1} Testing Devices ↓ DRC INSIGHT Secure App ↓ COS Service Device ↓ Local Content Delivery ↓ Fallback → DRC Cloud Supported Platforms Testing Devices: Windows, macOS, Linux, iPadOS, ChromeOS COS Service Device: Windows, macOS, Linux DRC states that testing devices can connect to any supported COS operating system type. :contentReference[oaicite:2]{index=2} Main COS Components management - configuration and management service relay - communicates with DRC backend services proxy - local proxy service contenthosting - serves local test content Important Design Notes Use COS for local content delivery and better testing consistency Do not design COS as an SMB or NFS share Registration with DRC is required for full functionality Chromebooks should register to the COS Configuration using the ORG Unit ID Next Page: Ubuntu COS Installation 2-DRC INSIGHT - Ubuntu COS Installation Purpose: Install the COS Service Device on Ubuntu and verify the core services start correctly. Prerequisites Ubuntu 22.04 or 24.04 Sudo/root access Internet access Installer package from the DRC portal Step 1 - Update Ubuntu sudo apt update && sudo apt upgrade -y Step 2 - Install Required Dependencies sudo apt install -y \ libxcb-xinerama0 \ libalt-getopt-complete-perl \ libxcb-icccm4 \ libxcb-image0 \ libxcb-keysyms1 \ libxcb-render-util0 \ libxcb-xkb1 \ libxkbcommon-x11-0 Critical: If these packages are missing, COS may install incompletely, fail silently, or launch only partial services. Step 3 - Fix /tmp Permissions sudo chmod 1777 /tmp ls -ld /tmp The COS installer writes temporary lock files to /tmp . Incorrect permissions can break the installer. Step 4 - Clean a Failed Installation (If Needed) sudo rm -rf /opt/CentralOffice sudo rm -f /tmp/coinstaller*.lock Step 5 - Run the Installer chmod +x silent-linux.sh sudo ./silent-linux.sh DRC recommends using the default install locations. During installation, save the setup URL displayed by the wizard, because it can be used to resume or complete the process. :contentReference[oaicite:3]{index=3} Step 6 - Verify Installation Files ls /opt/CentralOffice Expected folders include: management relay proxy contenthosting setup Step 7 - Verify Running Services sudo /opt/CentralOffice/node \ /opt/CentralOffice/setup/node_modules/forever/bin/forever -p /opt/CentralOffice list Expected: management relay proxy contenthosting Step 8 - Check Active Ports sudo ss -tulnp | grep node DRC training shows COS uses a base port, commonly 55222, and assigns consecutive ports from there. In this deployment, the active ports observed were 55223, 55224, and 55225. :contentReference[oaicite:4]{index=4} Step 9 - Test Local Response curl -I http://127.0.0.1:55223 Expected Result: HTTP 404 on the root path is acceptable. It confirms the service is alive even if the root URL is not the correct application route. Next Page: DRC Portal Configuration 3-DRC INSIGHT - DRC Portal Configuration for COS Purpose: Complete registration and configure Content Management and Content Hosting correctly. Step 1 - Complete COS Registration Open the setup URL generated during installation and complete the COS registration in the DRC portal. Important: If registration is not completed, relay and contenthosting may fail and devices may not be able to connect. Step 2 - Content Management Enable Content Management → ON Select required testing programs Select required accommodations Click Update Configuration DRC states that Content Management is used to choose the testing content and accommodations hosted by the COS Service Device. :contentReference[oaicite:5]{index=5} Step 3 - Content Hosting Enable Content Hosting → ON Load Balancer → OFF for a single COS server Shared Content Location → OFF for a single COS server Do Not Use Shared Content for a Single Server: Shared content is only appropriate in multi-server designs with a real shared storage architecture. Step 4 - Proxy Settings Use Proxy Host or Restricted Proxy → OFF unless explicitly required For a normal single COS Service Device deployment, avoid unnecessary proxy layers unless your network design specifically requires them. Step 5 - Add Testing Programs and Sites Use the Locations section to add the relevant testing programs and school or district site assignments to the COS Configuration. :contentReference[oaicite:6]{index=6} Step 6 - Save and Validate Click Update Configuration Wait for the configuration to turn green before testing DRC advises verifying that the configuration is green before students start testing each day. :contentReference[oaicite:7]{index=7} Next Page: Chromebook Deployment 4-DRC INSIGHT - Chromebook Deployment SOP Purpose: Deploy the DRC INSIGHT app to Chromebooks and prepare them for testing with COS. Step 1 - Install DRC INSIGHT Deploy the DRC INSIGHT app through Google Admin Console. DRC states that ChromeOS installations are deployed using Google Admin. :contentReference[oaicite:8]{index=8} Step 2 - Register Each Chromebook Launch DRC INSIGHT Select Assign Device to ORG Unit Enter the COS ORG Unit ID Select Register DRC training identifies ORG Unit assignment as a supported method for registering testing devices to a COS Configuration. :contentReference[oaicite:9]{index=9} Step 3 - Chromebook Settings Required by DRC Display Size → 100% Keyboard → US English DRC specifically states that ChromeOS display size should be 100%, and the keyboard should be US English so quotation marks and apostrophes display correctly. :contentReference[oaicite:10]{index=10} Step 4 - Same Network Requirement Keep Chromebooks on the same network path as the COS server during testing for reliable access to local content. Step 5 - Run Readiness Checks Use the DRC INSIGHT readiness and system checks before testing begins. DRC states the System Readiness Check helps confirm that each testing device meets minimum requirements and is operating properly before testing. :contentReference[oaicite:11]{index=11} Recommended Best Practices Disable or reduce unnecessary background apps Pause automatic updates during the testing window Validate registration before test day Next Page: Validation and Testing 5-DRC INSIGHT - Validation and Testing Purpose: Confirm COS is functioning properly before live testing begins. Step 1 - Verify Services sudo /opt/CentralOffice/node \ /opt/CentralOffice/setup/node_modules/forever/bin/forever -p /opt/CentralOffice list Confirm all four are running: management relay proxy contenthosting Step 2 - Verify Ports sudo ss -tulnp | grep node Step 3 - Validate Local HTTP Response curl -I http://127.0.0.1:55223 curl -I http://127.0.0.1:55224 curl -I http://127.0.0.1:55225 An HTTP 404 response at the root path can still indicate a healthy service. Step 4 - Validate Content Directory ls -lah /opt/CentralOffice/content_fs The local content directory should populate after configuration is updated and content is downloaded. Step 5 - Confirm DRC Portal Status Configuration shows green Correct test content is selected Content Hosting is enabled Step 6 - Validate Chromebook Access Confirm device is registered to the correct ORG Unit Confirm Chromebook settings are correct Open DRC INSIGHT and verify successful registration and readiness Pre-Test Validation Checklist [ ] COS registered with DRC [ ] All COS services running [ ] Node ports listening [ ] Local content downloaded [ ] Portal configuration green [ ] Chromebooks registered to ORG Unit [ ] Chromebook display size 100% [ ] Chromebook keyboard US English Next Page: Troubleshooting 6-DRC INSIGHT - COS Troubleshooting Purpose: Resolve the most common Linux COS installation, service, and Chromebook issues. Issue 1 - Installer Fails or Stops Early Symptoms: No usable COS services Installer exits quickly Lock file or temp errors Fix: sudo chmod 1777 /tmp sudo rm -rf /opt/CentralOffice sudo rm -f /tmp/coinstaller*.lock sudo ./silent-linux.sh Issue 2 - Only Proxy Runs Symptoms: forever list shows only proxy relay and contenthosting do not stay running Likely Cause: Registration not completed Configuration not generated correctly Fix: Complete the setup URL from installation Restart COS services Issue 3 - Relay Heartbeat Error Error: CO API - Heartbeat returned undefined Likely Cause: Registration incomplete DRC API not reachable Upstream filtering or SSL inspection interfering Fix: Complete registration Verify internet connectivity Allow required DRC URLs and ports Issue 4 - Services Show "active (exited)" This can be normal. COS uses forever to launch Node services in the background, so the systemd wrapper may show as exited while the Node processes continue running. Use this command as the truth source: sudo /opt/CentralOffice/node \ /opt/CentralOffice/setup/node_modules/forever/bin/forever -p /opt/CentralOffice list Issue 5 - Chromebooks Cannot Connect Check: ORG Unit ID entered correctly Chromebook can reach COS network path Display size is 100% Keyboard is US English Portal configuration is green Issue 6 - Content Does Not Download Check: Content Management enabled Content Hosting enabled Shared Content OFF for single server Correct test programs selected Useful Commands # Check runtime services sudo /opt/CentralOffice/node \ /opt/CentralOffice/setup/node_modules/forever/bin/forever -p /opt/CentralOffice list # Check ports sudo ss -tulnp | grep node # Check content directory ls -lah /opt/CentralOffice/content_fs # Follow content hosting logs sudo journalctl -u centralofficecontenthosting -f # Follow relay logs sudo journalctl -u centralofficerelay -f Support Contacts DRC PA Customer Service: (800) 451-7849 DRC Status Page: https://status.drcedirect.com/PA End of Troubleshooting SOP DRC INSIGHT – Central Office Services (COS) Installation & Troubleshooting (Full) Audience: IT Administrators / MSP Engineers Platform: Ubuntu 22.04 / 24.04 + Chromebook environments Overview DRC INSIGHT Central Office Services (COS) is a local content hosting system that allows testing devices to retrieve assessment content from a local server instead of the internet. Key Concept: COS is a service-based HTTP system, NOT a file share. Devices do NOT access /opt/CentralOffice directly. Architecture Testing Devices ↓ DRC INSIGHT App ↓ COS Server ↓ Local Content (/opt/CentralOffice/content_fs) ↓ Fallback → DRC Cloud Installation (Ubuntu) 1. Update System sudo apt update && sudo apt upgrade -y 2. Install Required Dependencies sudo apt install -y \ libxcb-xinerama0 \ libalt-getopt-complete-perl \ libxcb-icccm4 \ libxcb-image0 \ libxcb-keysyms1 \ libxcb-render-util0 \ libxcb-xkb1 \ libxkbcommon-x11-0 Critical: Missing dependencies will cause silent installer failures or services crashing. 3. Fix /tmp Permissions sudo chmod 1777 /tmp ls -ld /tmp 4. Clean Previous Install sudo rm -rf /opt/CentralOffice sudo rm -f /tmp/coinstaller*.lock 5. Run Installer chmod +x silent-linux.sh sudo ./silent-linux.sh Registration (REQUIRED) Important: You MUST open the setup URL generated during installation. This step registers COS with DRC and enables services. If skipped: Relay fails Contenthosting fails Devices cannot connect Validate Services Check Running Services sudo /opt/CentralOffice/node \ /opt/CentralOffice/setup/node_modules/forever/bin/forever -p /opt/CentralOffice list Expected: management relay proxy contenthosting Check Ports sudo ss -tulnp | grep node Typical Ports: 55223 – Content Hosting 55224 – Relay 55225 – Proxy Test Service curl -I http://127.0.0.1:55223 HTTP 404 is normal and confirms the service is running. DRC Portal Configuration Content Management Enable Content Management → ON Select tests (PSSA, Keystone, CDT) Click Update Configuration Content Hosting Enable Content Hosting → ON Load Balancer → OFF Shared Content Location → OFF Shared content should only be used in multi-server environments. Proxy Use Proxy → OFF Chromebook Configuration Install App Deploy DRC INSIGHT via Google Admin Console Register Device Launch App → Assign Device to ORG Unit → Enter ORG ID Required Settings Display Scale = 100% Keyboard = US English Same network as COS Network Requirements Allowlist *.drcedirect.com Ports 80 (HTTP) 443 (HTTPS) Internal COS Ports sudo ufw allow 55223/tcp sudo ufw allow 55224/tcp sudo ufw allow 55225/tcp Content Validation ls -lah /opt/CentralOffice/content_fs sudo journalctl -u centralofficecontenthosting -f Troubleshooting Installer Fails sudo chmod 1777 /tmp sudo rm -rf /opt/CentralOffice sudo ./silent-linux.sh Only Proxy Running Cause: COS not registered Heartbeat Error CO API - Heartbeat returned undefined Fix: Complete setup URL Verify internet access Allow DRC domains Chromebooks Cannot Connect Verify ORG ID Check network Confirm ports open Validation Checklist [ ] COS services running [ ] Ports listening [ ] Content downloading [ ] Portal configured correctly [ ] Chromebooks registered Key Takeaways COS is NOT a file server Uses HTTP endpoints Registration is required Content hosting must be enabled End of SOP