Index: /branches/amp_4_0/platform/tools/container/.env =================================================================== --- /branches/amp_4_0/platform/tools/container/.env (revision 2885) +++ /branches/amp_4_0/platform/tools/container/.env (working copy) @@ -48,3 +48,19 @@ # This MUST match the URL in your browser to avoid Grafana auth loops. # Uncomment to override auto-detection (e.g. for specific domain or IP). # AMP_DOMAIN_OR_IP=192.168.162.139 + +# --- Docker Image Versions --- +# These can be overridden to use different image versions. +# Defaults are set in manage_amp.sh if not specified here. +# TAG_OPENSEARCH=3.4.0 +# TAG_PGBOUNCER=v1.25.1-p0 +# TAG_GRAFANA=11.5.0 +# TAG_TELEGRAF=1.36.4 +# TAG_LOGSTASH=8.9.0 +# TAG_NGINX=1.25.3-alpine # Custom nginx with GUI +# TAG_TIMESCALEDB=latest-pg14 # Custom Patroni build +# TAG_OS_DASHBOARDS=3.4.0 +# TAG_ETCD=v3.5.17 +# TAG_HAPROXY=3.3-alpine +# TAG_AMP_CORE=1.0.0 + Index: /branches/amp_4_0/platform/tools/container/DEPLOYMENT.md =================================================================== --- /branches/amp_4_0/platform/tools/container/DEPLOYMENT.md (revision 2903) +++ /branches/amp_4_0/platform/tools/container/DEPLOYMENT.md (working copy) @@ -1,34 +1,46 @@ # AMP High Availability (HA) Cluster Deployment Guide -This guide describes how to deploy the Array Management Platform (AMP) on a 3-node (or larger) Docker Swarm cluster using the `manage_amp.sh` automation script. +This guide describes how to deploy the Array Management Platform (AMP) on a 3-node (or larger) Docker Swarm cluster. -## 1. Prerequisites +--- -### Hardware +## Part 1: Common Prerequisites (Both Online and Offline) -* **3 Nodes** (Physical or Virtual Machines) -* **OS**: Rocky Linux 9 / RHEL 9 (Recommended) -* **Resources**: Minimum 8GB RAM, 4 vCPUs per node. +### 1.1 Hardware Requirements -### Network +| Requirement | Specification | +|-------------|--------------| +| Nodes | 3+ (Physical or Virtual Machines) | +| OS | Rocky Linux 9 / RHEL 9 | +| RAM | Minimum 8GB per node | +| CPU | Minimum 4 vCPUs per node | -* All nodes must be on the same LAN/VLAN. -* Static IPs are recommended for stability. +### 1.2 Network Requirements -### Firewall & System Tuning (On ALL Nodes) +* All nodes must be on the same LAN/VLAN +* Static IPs recommended for stability +* Internet access required on **build machine only** (for online deployment) -OpenSearch requires increased virtual memory, and Docker Swarm needs specific ports open. Run on each node: +### 1.3 Build Machine Requirements (Online Deployment Only) +* Node.js v20+ (npm) - Required for GUI compilation +* Docker installed and running +* Internet access + +--- + +## Part 2: Online Deployment + +Use this section if your deployment machine has **internet access**. + +### Step 1: System Preparation (All Nodes) + +Run on **each node** to configure firewall and system settings: + ```bash ./manage_amp.sh system_tune ``` -This command automatically: - -* Sets `vm.max_map_count=262144` (persisted across reboots) -* Creates required log directories (`/var/log/amp/opensearch`) -* Configures all required firewall ports for Docker Swarm and AMP services -
Manual firewall configuration (if needed) @@ -40,340 +52,284 @@ firewall-cmd --add-port=4789/udp --permanent # AMP Service Ports -firewall-cmd --add-port=80/tcp --permanent # HTTP -firewall-cmd --add-port=443/tcp --permanent # HTTPS -firewall-cmd --add-port=5000/tcp --permanent # Local Registry -firewall-cmd --add-port=5432/tcp --permanent # Database (HAProxy/PGBouncer) -firewall-cmd --add-port=5433/tcp --permanent # Database (Direct Patroni) -firewall-cmd --add-port=2379-2380/tcp --permanent # Etcd -firewall-cmd --add-port=8008/tcp --permanent # Patroni API -firewall-cmd --add-port=9200/tcp --permanent # OpenSearch REST (Inter-node & Dashboards) -firewall-cmd --add-port=9300/tcp --permanent # OpenSearch Transport (Cluster) -firewall-cmd --add-port=5601/tcp --permanent # OpenSearch Dashboards -firewall-cmd --add-port=3000/tcp --permanent # Grafana -firewall-cmd --add-port=514/tcp --permanent # Logstash Syslog TCP -firewall-cmd --add-port=514/udp --permanent # Logstash Syslog UDP +firewall-cmd --add-port=80/tcp --permanent +firewall-cmd --add-port=443/tcp --permanent +firewall-cmd --add-port=5000/tcp --permanent +firewall-cmd --add-port=5432/tcp --permanent +firewall-cmd --add-port=9200/tcp --permanent +firewall-cmd --add-port=9300/tcp --permanent -# Reload firewall-cmd --reload ```
-> ⚠️ **WARNING**: Do NOT manually add Docker interfaces (`docker0`, `docker_gwbridge`) to firewalld zones. Docker manages these interfaces itself. Manual zone assignments cause `ZONE_CONFLICT` errors that prevent Docker from starting. +> ⚠️ **WARNING**: Do NOT manually add Docker interfaces (`docker0`, `docker_gwbridge`) to firewalld zones. ---- +### Step 2: Docker Swarm Setup -## 2. Docker Swarm Setup +**On Node 1 (Manager):** -1. **Initialize Swarm on Manager Node (Node 1)**: +```bash +docker swarm init --advertise-addr +``` - ```bash - docker swarm init --advertise-addr - ``` +Copy the join command output. - *Copy the "docker swarm join" command output.* +**On Node 2 and Node 3:** -2. **Join Worker Nodes (Node 2, Node 3)**: - Run the command copied from step 1 on the other nodes: +```bash +docker swarm join --token :2377 +``` - ```bash - docker swarm join --token :2377 - ``` +**On Node 1 - Promote workers to managers:** -3. **Rename Nodes (Optional but Recommended)**: - Assign readable hostnames if not already set (e.g., `amp-node-1`, `amp-node-2`). The script uses Docker Hostnames. +```bash +docker node ls +docker node promote +docker node promote +``` - To rename a node (run on the respective node): +### Step 3: Configure Virtual IP (Optional but Recommended) - ```bash - hostnamectl set-hostname - ``` +**On Node 1:** -4. **Promote Workers to Managers**: - Promote the newly joined worker nodes to managers, as all nodes function as managers in this setup. +```bash +./manage_amp.sh vip --vip --priority 101 +``` - On **Node 1 (Manager)**: +**On Node 2:** - ```bash - docker node ls - docker node promote - ``` +```bash +./manage_amp.sh vip --vip --priority 100 +``` ---- +### Step 4: Build and Push Images -## 3. Offline Deployment (Air-Gapped Environments) +Navigate to the container directory and build: -If you've created an offline bundle using `./manage_amp.sh bundle` on a build machine with internet access, follow these steps: +```bash +cd platform/tools/container/ +./manage_amp.sh build +``` -### Prerequisites +> **Note**: This compiles the Angular GUI and builds custom Docker images. Ensure Node.js v20+ is installed. -1. **Complete Section 1**: Ensure all prerequisites (hardware, network, firewall rules) are met on all nodes. -2. **Complete Section 2**: Set up Docker Swarm cluster (init, join workers, promote to managers). +### Step 5: Deploy the Stack -### On the Offline Target Machine (Manager Node) +```bash +./manage_amp.sh deploy --auto +``` -1. **Transfer Files**: Copy these files to the offline machine: - * `amp_offline_bundle.tar.gz` - * `tar-bootstrap.rpm` (for minimal Rocky Linux installations) +### Step 6: Post-Deployment Configuration -2. **Install tar** (if not present): +Run these commands **once** after first deployment: - ```bash - rpm -ivh tar-bootstrap.rpm - ``` +```bash +# Initialize OpenSearch security +./manage_amp.sh security_init -3. **Extract Bundle**: +# Create Grafana database +./manage_amp.sh create_grafana_db +# Import dashboards and index templates +./manage_amp.sh configurator +``` + +### Step 7: Verify Deployment + +```bash +docker service ls +``` + +Access the GUI at `https:///` or `https:///` + +--- + +## Part 3: Offline Deployment (Air-Gapped) + +Use this section for environments **without internet access**. + +### Phase A: Create Offline Bundle (Build Machine with Internet) + +**On a machine with internet access:** + +1. **Prepare environment:** + ```bash - tar -xf amp_offline_bundle.tar.gz - cd amp_offline_bundle + cd platform/tools/container/ + ./install_prerequisites.sh ``` -4. **Load Offline Bundle**: +2. **Create the offline bundle:** ```bash - ./manage_amp.sh load_offline + ./manage_amp.sh bundle ``` - This installs all dependencies (Docker, rsync, keepalived, Java, Python) and loads Docker images into the local registry. +3. **Transfer to offline environment:** -5. **Continue with Standard Deployment**: After `load_offline` completes successfully, **jump to Section 4 below** (starting from Step 0: Configure VIP). + Copy these files to your air-gapped target: + * `amp_offline_bundle.tar.gz` + * `tar-bootstrap.rpm` --- -## 4. Standard Online Deployment +### Phase B: Deploy on Offline Target -All deployment actions are handled by the `manage_amp.sh` script on **Node 1 (Manager)**. +**On the offline target machines:** -### Step 0: Configure Virtual IP (VIP) for HA +#### Step 1: Install tar (if needed) -To ensure High Availability, configure a Floating VIP that will automatically failover between nodes. +```bash +rpm -ivh tar-bootstrap.rpm +``` -On **Node 1 (Master)**: +#### Step 2: Extract Bundle ```bash -./manage_amp.sh vip --vip --priority 101 +tar -xf amp_offline_bundle.tar.gz +cd amp_offline_bundle ``` -On **Node 2 (Backup)**: +#### Step 3: System Preparation (All Nodes) ```bash -./manage_amp.sh vip --vip --priority 100 +./manage_amp.sh system_tune ``` -*This command configures Keepalived and updates your configuration to use this VIP.* +#### Step 4: Docker Swarm Setup -### Step 1: Prepare Environment +**On Node 1 (Manager):** -Navigate to the container directory: - ```bash -cd container/ +docker swarm init --advertise-addr ``` -Check `.env` file (Optional). The defaults are usually sufficient. You mainly only need to set passwords if you want non-defaults. +**On Node 2 and Node 3:** ```bash -vi .env +docker swarm join --token :2377 ``` -### Step 2: Build & Push Images (First Time or Updates) +**Promote workers to managers:** -This step pulls the required images from the internet (or loads them) and pushes them to the local registry so all Swarm nodes can access them. - ```bash -./manage_amp.sh build +docker node promote +docker node promote ``` -*This may take a while depending on your internet connection.* +#### Step 5: Load Offline Bundle -### Step 3: Auto-Configure & Deploy - -Run the deploy command with the `--auto` flag. This will: - -1. Detect all Swarm nodes. -2. Auto-populate IPs in `.env`. -3. Generate the `stack.yml` dynamically (adding Etcd/DB services for each node). -4. Deploy the stack. - ```bash -./manage_amp.sh deploy --auto +./manage_amp.sh load_offline ``` -### Step 4: Setup Certificates (Automated) +This installs Docker, loads images, and pushes to the local registry. -The deployment script (`deploy --auto`) automatically checks for and triggers certificate generation if they are missing. +#### Step 6: Configure Virtual IP (Optional) -*No manual action required.* - -### Step 5: Initialize Security (First Time Only) - -Initialize the OpenSearch security index. - ```bash -./manage_amp.sh security_init +./manage_amp.sh vip --vip --priority 101 # Node 1 +./manage_amp.sh vip --vip --priority 100 # Node 2 ``` -### Step 6: Initialize Grafana DB (First Time Only) +#### Step 7: Deploy the Stack -Create the Grafana database user and schema in the HA Postgres cluster. - ```bash -./manage_amp.sh create_grafana_db +./manage_amp.sh deploy --auto ``` -### Step 7: Configure OpenSearch Dashboards (First Time Only) +#### Step 8: Post-Deployment Configuration -Import Dashboards, Index Patterns, and Index Templates (ISM Policies). - ```bash +./manage_amp.sh security_init +./manage_amp.sh create_grafana_db ./manage_amp.sh configurator ``` -### Step 8: Create CLI Users (Optional) +#### Step 9: Verify Deployment -Create users who will access the AMP CLI via SSH: - ```bash -# Create a user with ca_shell (CLI only) -sudo useradd -m -s /ca/bin/ca_shell amp_operator -sudo passwd amp_operator - -# Create a user with bash (full access) -sudo useradd -m -s /bin/bash amp_admin -sudo passwd amp_admin - -# Add users to docker group (required for ca_shell) -sudo usermod -aG docker amp_operator +docker service ls ``` -> **Note**: The deploy script automatically installs `/ca/bin/ca_shell` wrapper and sets the SELinux context. - --- -## 4. Verification +## Part 4: Verification -### Check Services +### Check All Services -Ensure all services are up and running (expected: `3/3` replicas for global services, `1/1` for others). - ```bash docker service ls ``` -### Verify HA / Failover +Expected: All services show correct replicas (e.g., `3/3` for global, `1/1` for replicated). -1. **Web Access**: Open `https:///` or `https:///`. You should see the AMP login. -2. **Database**: Connect to Port `5432` on any node. It routes to the current Primary. +### Access the UI - ```bash - psql -h 127.0.0.1 -p 5432 -U amp_ts_user amp_ts - ``` +* **Web GUI**: `https:///` or `https:///` +* **Dashboards**: `https:///dashboards/` +* **Grafana**: `https:///monitoring/` -3. **Failover Test**: Reboot a node inside the cluster. - * **Result**: The cluster should remain operational. - * Services will reschedule to remaining nodes. - * Database leadership will failover automatically via Patroni/Etcd. +### Test HA Failover +1. Reboot one node +2. Verify cluster remains operational +3. Services reschedule automatically + --- -## 5. Troubleshooting +## Part 5: Troubleshooting -* **Logs**: `docker service logs -f amp_` -* **Manual Config Update**: If you add a new node to the swarm, re-run: +| Symptom | Cause | Solution | +|---------|-------|----------| +| Service shows `0/1` replicas | Image not found | Run `./manage_amp.sh build` then `deploy --auto` | +| `502 Bad Gateway` | Backend service down | Check `docker service logs amp_` | +| `ZONE_CONFLICT` Docker crash | Firewalld misconfiguration | Remove Docker interfaces from manual zones | +| OpenSearch security not working | Security not initialized | Run `./manage_amp.sh security_init` | - ```bash - ./manage_amp.sh deploy --auto - ``` +### Useful Commands -### Common Issues +```bash +# View service logs +docker service logs -f amp_ -| Symptom | Cause | Solution | -|---------|-------|----------| -| `502 Bad Gateway` on some requests | Dashboards not running on all nodes | Ensure `mode: global` in `stack.yml` | -| `ZONE_CONFLICT` Docker crash | Manual firewalld zone assignment | Remove Docker interfaces from manual zones | -| `invalid mount config` | Missing log directory on node | Create `/var/log/amp/opensearch` on all nodes | -| `ECONNREFUSED` to OpenSearch | Firewall blocking port 9200 | Open port 9200 on all nodes | +# Force service update +docker service update --force amp_ +# Redeploy after changes +./manage_amp.sh deploy --auto +``` + --- ## Appendix A: Docker Images -The following Docker images are bundled/used by AMP: - | Service | Image | Description | |---------|-------|-------------| +| nginx | Custom build (GUI) | Reverse proxy with AMP GUI | +| timescaledb | Custom build (Patroni) | Time-series database with HA | | opensearch | `opensearchproject/opensearch` | Search and analytics engine | | opensearch-dashboards | `opensearchproject/opensearch-dashboards` | Visualization UI | -| timescaledb | Custom build (Patroni) | Time-series database with HA | -| pgbouncer | `edoburu/pgbouncer` | Connection pooling | | grafana | `grafana/grafana` | Monitoring dashboards | -| telegraf | `telegraf` | Metrics collection agent | | logstash | `opensearchproject/logstash-oss-with-opensearch-output-plugin` | Log ingestion | -| nginx | `nginx` | Reverse proxy | +| telegraf | `telegraf` | Metrics collection agent | | etcd | `quay.io/coreos/etcd` | Distributed key-value store | | haproxy | `haproxy` | Database load balancer | -| registry | `registry` | Local Docker registry | -| busybox | `busybox` | Utility container | -| rocky | `rockylinux` | Base OS image | +| pgbouncer | `edoburu/pgbouncer` | Connection pooling | --- -## Appendix B: RPM Packages (Offline Bundle) +## Appendix B: Default Ports -The offline bundle includes these packages and their dependencies: - -| Package | Purpose | -|---------|---------| -| docker-ce, docker-ce-cli, containerd.io | Docker runtime | -| docker-buildx-plugin, docker-compose-plugin | Docker plugins | -| keepalived | VIP failover (VRRP) | -| rsync | File synchronization | -| python3 | Scripting | -| java-17-openjdk | OpenSearch security tools | -| tar | Archive extraction | -| openssl, httpd-tools | Certificate generation | -| curl, jq | API calls and JSON parsing | -| bind-utils | DNS tools (dig, nslookup) | -| iputils | Network tools (ping) | -| net-tools | Network debugging (netstat, ifconfig) | - ---- - -## Appendix C: Service Deployment Modes - -| Service | Mode | Port | Rationale | -|---------|------|------|-----------| -| nginx | global | 80, 443 | Web access on every node | -| opensearch-dashboards | global | 5601 | Local proxy access | -| grafana | global | 3000 | Local proxy access | -| logstash | global | 514 | Syslog on all nodes | -| telegraf | global | - | Docker monitoring per node | -| haproxy | global | 5432 | Database LB per node | -| pgbouncer | global | - | Connection pooling | -| opensearch | replicated (3) | 9200, 9300 | Stateful cluster | -| timescaledb | replicated (3) | 5433 | Patroni HA cluster | -| etcd | replicated (3) | 2379-2380 | Raft consensus | - ---- - -## Appendix D: Default Ports - -| Port | Protocol | Service | Notes | -|------|----------|---------|-------| -| 80 | TCP | Nginx (HTTP) | Redirects to HTTPS | -| 443 | TCP | Nginx (HTTPS) | Web UI entry point | -| 514 | TCP/UDP | Logstash | Syslog ingestion | -| 2377 | TCP | Docker Swarm | Cluster management | -| 2379-2380 | TCP | Etcd | Cluster coordination | -| 3000 | TCP | Grafana | Monitoring UI | -| 4789 | UDP | Docker Overlay | Container networking | -| 5000 | TCP | Registry | Local image storage | -| 5432 | TCP | HAProxy | Database (via LB) | -| 5433 | TCP | TimescaleDB | Direct Patroni access | -| 5601 | TCP | Dashboards | OpenSearch UI | -| 7946 | TCP/UDP | Docker Swarm | Node communication | -| 8008 | TCP | Patroni | Health API | -| 9200 | TCP | OpenSearch | REST API | -| 9300 | TCP | OpenSearch | Cluster transport | +| Port | Service | Notes | +|------|---------|-------| +| 80/443 | Nginx | Web UI entry | +| 514 | Logstash | Syslog ingestion | +| 2377 | Docker Swarm | Cluster management | +| 3000 | Grafana | Monitoring UI | +| 5000 | Registry | Local image storage | +| 5432 | HAProxy | Database (via LB) | +| 5601 | Dashboards | OpenSearch UI | +| 9200 | OpenSearch | REST API | Index: /branches/amp_4_0/platform/tools/container/OFFLINE_GUIDE.md =================================================================== --- /branches/amp_4_0/platform/tools/container/OFFLINE_GUIDE.md (revision 2885) +++ /branches/amp_4_0/platform/tools/container/OFFLINE_GUIDE.md (working copy) @@ -11,6 +11,7 @@ * **Tools**: * `dnf` (Standard package manager) * `docker` (Installed and running) + * `npm` (Node.js) - Required for compiling the Angular GUI **Setup Build Machine**: Before creating the bundle, you must ensure Docker is installed and running on this machine (so it can pull the images). Index: /branches/amp_4_0/platform/tools/container/README.md =================================================================== --- /branches/amp_4_0/platform/tools/container/README.md (revision 2885) +++ /branches/amp_4_0/platform/tools/container/README.md (working copy) @@ -62,8 +62,11 @@ ./manage_amp.sh deploy ``` - *This builds images, pushes them to the local registry, and deploys the `amp` stack.* + *This builds images (including compiling the Angular GUI into the nginx image), pushes them to the local registry, and deploys the `amp` stack.* + > [!NOTE] + > **GUI Build Requirement**: The `build` command compiles the Angular GUI from `src/webui/webui/htdocs/new/src/gui/`. Ensure Node.js (npm) is installed on the build machine. + 5. **Initialize Security**: ```bash Index: /branches/amp_4_0/platform/tools/container/manage_amp.sh =================================================================== --- /branches/amp_4_0/platform/tools/container/manage_amp.sh (revision 2903) +++ /branches/amp_4_0/platform/tools/container/manage_amp.sh (working copy) @@ -13,7 +13,8 @@ export TAG_GRAFANA="${TAG_GRAFANA:-11.5.0}" export TAG_TELEGRAF="${TAG_TELEGRAF:-1.36.4}" export TAG_LOGSTASH="${TAG_LOGSTASH:-8.9.0}" -export TAG_NGINX="${TAG_NGINX:-stable-alpine}" +export TAG_NGINX="${TAG_NGINX:-1.25.3-alpine}" # Base nginx version with GUI +export TAG_TIMESCALEDB="${TAG_TIMESCALEDB:-latest-pg14}" # Base timescaledb version with Patroni export TAG_OS_DASHBOARDS="${TAG_OS_DASHBOARDS:-3.4.0}" export TAG_ETCD="${TAG_ETCD:-v3.5.17}" export TAG_HAPROXY="${TAG_HAPROXY:-3.3-alpine}" @@ -29,19 +30,19 @@ # Format: "service_name:upstream_image" IMAGES=( "opensearch:opensearchproject/opensearch:${TAG_OPENSEARCH}" - "timescaledb:custom_build" # Uses :latest by default for custom build + "timescaledb:${TAG_TIMESCALEDB}" # Custom Patroni build "pgbouncer:edoburu/pgbouncer:${TAG_PGBOUNCER}" "grafana:grafana/grafana:${TAG_GRAFANA}" "telegraf:telegraf:${TAG_TELEGRAF}" "logstash:opensearchproject/logstash-oss-with-opensearch-output-plugin:${TAG_LOGSTASH}" - "nginx:nginx:${TAG_NGINX}" + "nginx:${TAG_NGINX}" # Custom build with GUI "opensearch-dashboards:opensearchproject/opensearch-dashboards:${TAG_OS_DASHBOARDS}" "etcd:quay.io/coreos/etcd:${TAG_ETCD}" "haproxy:haproxy:${TAG_HAPROXY}" "registry:registry:${TAG_REGISTRY}" "busybox:busybox:${TAG_BUSYBOX}" "rocky:rockylinux:${TAG_ROCKY}" - "amp-core:custom_build" # Backend + WebUI Agent + "amp-core:${TAG_AMP_CORE}" # Backend + WebUI Agent ) # REPO_ROOT Resolution @@ -845,8 +846,11 @@ # Determine Tag and Source Image if [ "$SERVICE_NAME" == "timescaledb" ]; then - TAG="latest" - SRC_IMAGE="amp/timescaledb:latest" + TAG="${TAG_TIMESCALEDB}" + SRC_IMAGE="amp/timescaledb:${TAG_TIMESCALEDB}" + elif [ "$SERVICE_NAME" == "nginx" ]; then + TAG="${TAG_NGINX}" + SRC_IMAGE="amp/nginx:${TAG_NGINX}" elif [ "$SERVICE_NAME" == "amp-core" ]; then TAG="$TAG_AMP_CORE" SRC_IMAGE="amp/amp-core:$TAG_AMP_CORE" @@ -867,7 +871,7 @@ elif [ "$SERVICE_NAME" == "timescaledb" ]; then echo " - Building Custom Image..." docker build -t "$LOCAL_TAG" -f "$SERVICES_DIR/timescaledb/Dockerfile.patroni" "$SERVICES_DIR/timescaledb" || { - echo " ❌ Build failed. In offline mode, ensure amp/timescaledb:latest is pre-loaded." + echo " ❌ Build failed. In offline mode, ensure amp/timescaledb:${TAG_TIMESCALEDB} is pre-loaded." continue } elif [ "$SERVICE_NAME" == "amp-core" ]; then @@ -920,6 +924,22 @@ echo " ❌ Build failed. Ensure binaries are present in $AMP_CORE_BIN." continue } + elif [ "$SERVICE_NAME" == "nginx" ]; then + echo " - Building nginx with GUI..." + NGINX_DIR="$SERVICES_DIR/nginx" + + # Build GUI if not already present + if [ ! -d "$NGINX_DIR/gui" ] || [ ! -f "$NGINX_DIR/gui/index.html" ]; then + build_gui || { + echo " ❌ GUI build failed. Cannot build nginx image." + continue + } + fi + + docker build -t "$LOCAL_TAG" "$NGINX_DIR" || { + echo " ❌ nginx build failed." + continue + } else echo " - Pulling $UPSTREAM" docker pull -q "$UPSTREAM" || { @@ -939,6 +959,101 @@ done } +build_gui() { + echo "--- Building Angular GUI ---" + GUI_SRC="$SCRIPT_DIR/../../../src/webui/webui/htdocs/new/src/gui" + GUI_DIST="$SERVICES_DIR/nginx/gui" + + # Resolve absolute path + GUI_SRC="$(cd "$GUI_SRC" 2>/dev/null && pwd)" || { + echo "❌ GUI source not found at $GUI_SRC" + return 1 + } + + # Check and install Node.js if missing or too old (Angular 21 requires v20.19+) + NEED_NODEJS=false + if ! command -v npm &> /dev/null; then + echo "⚠️ Node.js not found. Installing..." + NEED_NODEJS=true + else + NODE_VERSION=$(node --version | sed 's/v//') + NODE_MAJOR=$(echo "$NODE_VERSION" | cut -d. -f1) + if [ "$NODE_MAJOR" -lt 22 ]; then + echo "⚠️ Node.js version $NODE_VERSION is too old. Installing v20 LTS..." + NEED_NODEJS=true + fi + fi + + if [ "$NEED_NODEJS" = true ]; then + if command -v dnf &> /dev/null; then + # Rocky/RHEL/Fedora - remove old packages first to avoid conflicts + echo "Removing old Node.js packages..." + sudo dnf remove -y nodejs npm nodejs-full-i18n 2>/dev/null || true + + echo "Adding NodeSource repository..." + curl -fsSL https://rpm.nodesource.com/setup_22.x | sudo bash - + + echo "Installing Node.js v22..." + sudo dnf install -y nodejs + elif command -v apt-get &> /dev/null; then + # Debian/Ubuntu - remove old packages first + echo "Removing old Node.js packages..." + sudo apt-get remove -y nodejs npm 2>/dev/null || true + + echo "Adding NodeSource repository..." + curl -fsSL https://deb.nodesource.com/setup_22.x | sudo bash - + + echo "Installing Node.js v22..." + sudo apt-get install -y nodejs + else + echo "❌ Unsupported OS. Install Node.js v22+ manually." + return 1 + fi + + # Verify installation + if ! command -v npm &> /dev/null; then + echo "❌ Node.js installation failed." + return 1 + fi + + NODE_VER=$(node --version) + echo "✅ Node.js $NODE_VER installed." + fi + + echo "Building GUI from: $GUI_SRC" + cd "$GUI_SRC" + + # Clean existing node_modules to ensure fresh install + rm -rf node_modules package-lock.json + + # Verify package.json has the xterm packages + echo "Checking package.json for @xterm packages..." + grep -E "@xterm/(xterm|addon-fit)" package.json || echo "⚠️ @xterm packages not found in package.json!" + + # Install dependencies + echo "Installing npm packages..." + npm install --legacy-peer-deps || { echo "❌ npm install failed"; return 1; } + + # Verify xterm packages were installed + if [ ! -d "node_modules/@xterm/xterm" ]; then + echo "❌ @xterm/xterm was not installed!" + echo "Checking if packages exist in npm..." + npm view @xterm/xterm version || echo "Package not found in npm registry" + return 1 + fi + echo "✅ @xterm packages installed successfully" + + npm run build || { echo "❌ npm build failed"; return 1; } + + # Copy dist to nginx service directory + rm -rf "$GUI_DIST" + mkdir -p "$GUI_DIST" + cp -r "$GUI_SRC/dist/gui/browser/"* "$GUI_DIST/" + + echo "✅ GUI built and copied to $GUI_DIST" + cd "$SCRIPT_DIR" +} + create_secrets() { echo "--- strict Checking/Creating Secrets ---" @@ -1400,8 +1515,8 @@ if [ "$SERVICE_NAME" == "timescaledb" ]; then echo " - Building Custom Image..." - retry_command "docker build -t amp/timescaledb:latest -f \"$SERVICES_DIR/timescaledb/Dockerfile.patroni\" \"$SERVICES_DIR/timescaledb\"" || exit 1 - IMAGE_LIST="amp/timescaledb:latest $IMAGE_LIST" + retry_command "docker build -t amp/timescaledb:${TAG_TIMESCALEDB} -f \"$SERVICES_DIR/timescaledb/Dockerfile.patroni\" \"$SERVICES_DIR/timescaledb\"" || exit 1 + IMAGE_LIST="amp/timescaledb:${TAG_TIMESCALEDB} $IMAGE_LIST" elif [ "$SERVICE_NAME" == "amp-core" ]; then echo " - Building amp-core Image (Backend + WebUI Agent)..." @@ -1450,6 +1565,15 @@ # Build the Docker image retry_command "docker build -t amp/amp-core:${TAG_AMP_CORE} \"$AMP_CORE_DIR\"" || exit 1 IMAGE_LIST="amp/amp-core:${TAG_AMP_CORE} $IMAGE_LIST" + elif [ "$SERVICE_NAME" == "nginx" ]; then + echo " - Building nginx with GUI..." + NGINX_DIR="$SERVICES_DIR/nginx" + + # Build GUI first + build_gui || exit 1 + + retry_command "docker build -t amp/nginx:${TAG_NGINX} \"$NGINX_DIR\"" || exit 1 + IMAGE_LIST="amp/nginx:${TAG_NGINX} $IMAGE_LIST" else echo " - Pulling $UPSTREAM" retry_command "docker pull -q $UPSTREAM" || exit 1 @@ -1466,7 +1590,7 @@ # Compress the tarball to save space (gzip) echo "Compressing image archive..." - gzip "$BUNDLE_DIR/images/amp_images.tar" + gzip -f "$BUNDLE_DIR/images/amp_images.tar" echo "✅ Images saved to $BUNDLE_DIR/images/amp_images.tar.gz" @@ -1612,8 +1736,11 @@ # Determine Tag and Source Image for custom builds if [ "$SERVICE_NAME" == "timescaledb" ]; then - TAG="latest" - SRC_IMAGE="amp/timescaledb:latest" + TAG="${TAG_TIMESCALEDB}" + SRC_IMAGE="amp/timescaledb:${TAG_TIMESCALEDB}" + elif [ "$SERVICE_NAME" == "nginx" ]; then + TAG="${TAG_NGINX}" + SRC_IMAGE="amp/nginx:${TAG_NGINX}" elif [ "$SERVICE_NAME" == "amp-core" ]; then TAG="$TAG_AMP_CORE" SRC_IMAGE="amp/amp-core:$TAG_AMP_CORE" Index: /branches/amp_4_0/platform/tools/container/services/nginx/Dockerfile =================================================================== --- /branches/amp_4_0/platform/tools/container/services/nginx/Dockerfile (nonexistent) +++ /branches/amp_4_0/platform/tools/container/services/nginx/Dockerfile (working copy) @@ -0,0 +1,12 @@ +FROM nginx:1.25.3-alpine + +# Remove default nginx static content +RUN rm -rf /usr/share/nginx/html/* + +# Copy pre-built Angular GUI +COPY gui/ /usr/share/nginx/html/ + +# Ensure proper permissions +RUN chown -R nginx:nginx /usr/share/nginx/html + +EXPOSE 80 443 Index: /branches/amp_4_0/platform/tools/container/stack.yml.template =================================================================== --- /branches/amp_4_0/platform/tools/container/stack.yml.template (revision 2903) +++ /branches/amp_4_0/platform/tools/container/stack.yml.template (working copy) @@ -117,7 +117,7 @@ - hostnet timescaledb: - image: ${REGISTRY:-127.0.0.1:5000}/amp/timescaledb:latest + image: ${REGISTRY:-127.0.0.1:5000}/amp/timescaledb:${TAG_TIMESCALEDB:-latest-pg14} deploy: mode: replicated replicas: ${AMP_REPLICAS:-3} Index: /branches/amp_4_0/src/webui/webui/htdocs/new/src/gui/package.json =================================================================== --- /branches/amp_4_0/src/webui/webui/htdocs/new/src/gui/package.json (revision 2885) +++ /branches/amp_4_0/src/webui/webui/htdocs/new/src/gui/package.json (working copy) @@ -10,6 +10,9 @@ "serve:ssr:gui": "node dist/gui/server/server.mjs" }, "private": true, + "engines": { + "node": ">=20.19.0" + }, "dependencies": { "@angular/animations": "^21.0.6", "@angular/common": "^21.0.6", @@ -22,7 +25,7 @@ "@angular/platform-server": "^21.0.6", "@angular/router": "^21.0.6", "@angular/ssr": "^21.0.4", - "@fortawesome/angular-fontawesome": "^2.0.1", + "@fortawesome/angular-fontawesome": "^4.0.0", "@fortawesome/fontawesome-svg-core": "^6.7.2", "@fortawesome/free-regular-svg-icons": "^6.7.2", "@fortawesome/free-solid-svg-icons": "^6.7.2", @@ -33,10 +36,10 @@ "ngx-echarts": "^20.0.1", "rxjs": "~7.8.0", "tslib": "^2.3.0", - "xterm": "^5.3.0", - "xterm-addon-fit": "^0.8.0", + "@xterm/xterm": "^5.5.0", + "@xterm/addon-fit": "^0.10.0", "zone.js": "~0.15.0", - "@ngxmc/datetime-picker": "20.1.0" + "@ngxmc/datetime-picker": "^20.1.0" }, "devDependencies": { "@angular/build": "^21.0.4", Index: /branches/amp_4_0/src/webui/webui/htdocs/new/src/gui/src/app/components/sub-components/web-console/web-console.ts =================================================================== --- /branches/amp_4_0/src/webui/webui/htdocs/new/src/gui/src/app/components/sub-components/web-console/web-console.ts (revision 2885) +++ /branches/amp_4_0/src/webui/webui/htdocs/new/src/gui/src/app/components/sub-components/web-console/web-console.ts (working copy) @@ -13,8 +13,8 @@ export class WebConsole implements OnInit, AfterViewInit, OnDestroy { @ViewChild('terminal', { static: false }) terminalElementRef!: ElementRef; - private terminal: import('xterm').Terminal | null = null; - private fitAddon: import('xterm-addon-fit').FitAddon | null = null; + private terminal: import('@xterm/xterm').Terminal | null = null; + private fitAddon: import('@xterm/addon-fit').FitAddon | null = null; private resizeObserver: ResizeObserver | null = null; title = 'Web Console'; info = 'Connecting...'; @@ -48,8 +48,8 @@ async ngAfterViewInit(): Promise { if (isPlatformBrowser(this.platformId)) { - const { Terminal } = await import('xterm'); - const { FitAddon } = await import('xterm-addon-fit'); + const { Terminal } = await import('@xterm/xterm'); + const { FitAddon } = await import('@xterm/addon-fit'); if (this.terminalElementRef?.nativeElement) { this.terminal = new Terminal({ Index: /branches/amp_4_0/src/webui/webui/htdocs/new/src/gui/src/styles.scss =================================================================== --- /branches/amp_4_0/src/webui/webui/htdocs/new/src/gui/src/styles.scss (revision 2885) +++ /branches/amp_4_0/src/webui/webui/htdocs/new/src/gui/src/styles.scss (working copy) @@ -1,7 +1,7 @@ /* You can add global styles to this file, and also import other style files */ @use '@angular/material' as mat; @import '@angular/material/prebuilt-themes/azure-blue.css'; -@import 'xterm/css/xterm.css'; +@import '@xterm/xterm/css/xterm.css'; @font-face { font-family: 'Roboto';