Planning build system simplification

triage/build-system-description.md

@@ -0,0 +1,203 @@
+# 4coder Build System Analysis
+
+## Overview
+
+The 4coder build system is a sophisticated multi-tier architecture designed to handle cross-platform compilation for Windows, Linux, and macOS. The system consists of three main components:
+
+1. **Meta-Build System**: A C++ program that generates platform-specific build commands
+2. **Platform-Specific Entry Scripts**: Shell/batch scripts that compile and execute the meta-build program
+3. **Custom Layer Build System**: Separate compilation process for the customizable editor layer
+
+## Architecture Deep Dive
+
+### 1. Meta-Build System (`code/bin/4ed_build.cpp`)
+
+The core of the build system is a 720-line C++ program that serves as a build configuration generator. Key features:
+
+- **Platform Detection**: Automatically detects Windows, Linux, or macOS
+- **Compiler Support**: Handles CL (MSVC), GCC, and Clang compilers
+- **Architecture Support**: Supports both x64 and x86 architectures
+- **Build Tiers**: Manages Demo vs Super build configurations
+- **Flag Management**: Centralized handling of compilation flags and options
+
+#### Key Data Structures:
+```cpp
+typedef u32 Platform_Code;
+enum { Platform_Windows, Platform_Linux, Platform_Mac };
+
+typedef u32 Compiler_Code;
+enum { Compiler_CL, Compiler_GCC, Compiler_Clang };
+
+typedef u32 Arch_Code;
+enum { Arch_X64, Arch_X86 };
+
+typedef u32 Tier_Code;
+enum { Tier_Demo, Tier_Super };
+```
+
+#### Build Flags System:
+```cpp
+enum {
+    OPTS = 0x1,          // Compiler warnings/options
+    LIBS = 0x2,          // Link libraries
+    ICON = 0x4,          // Windows icon resource
+    SHARED_CODE = 0x8,   // Build as shared library
+    DEBUG_INFO = 0x10,   // Debug symbols
+    OPTIMIZATION = 0x20, // Optimization level
+    SUPER = 0x40,        // Super tier features
+    INTERNAL = 0x80,     // Internal build flags
+    SHIP = 0x100,        // Shipping build
+};
+```
+
+### 2. Platform-Specific Entry Scripts
+
+Each platform has dedicated build scripts that:
+1. Compile the meta-build system (`4ed_build.cpp`)
+2. Execute the compiled meta-build program with appropriate flags
+3. Handle platform-specific compiler setup
+
+#### macOS (`code/bin/build-mac-dev.sh`):
+```bash
+#!/bin/bash
+g++ -std=c++11 -DDEV_BUILD -Wno-write-strings -Wno-comment -Wno-null-dereference -Wno-logical-op-parentheses -Wno-switch -o ../build/4ed_build 4ed_build.cpp
+../build/4ed_build $1
+```
+
+#### Linux (`code/bin/build-linux.sh`):
+```bash
+#!/bin/bash
+g++ -g -std=c++11 -DDEV_BUILD -Wno-write-strings -Wno-comment -Wno-null-dereference -Wno-logical-op-parentheses -Wno-switch -o ../build/4ed_build 4ed_build.cpp
+../build/4ed_build $1
+```
+
+#### Windows (`code/bin/build.bat`):
+```batch
+@echo off
+IF NOT EXIST ..\build mkdir ..\build
+pushd ..\build
+cl /I..\code /DDEV_BUILD /std:c++11 /Zi /Fe:4ed_build.exe ..\code\bin\4ed_build.cpp /link /INCREMENTAL:NO /RELEASE > NUL
+4ed_build.exe %1
+popd
+```
+
+### 3. Custom Layer Build System
+
+The custom layer (editor behavior/commands) uses a sophisticated multi-stage build process:
+
+#### Stage 1: Preprocessing with Meta Macros
+```bash
+clang++ -I../code -I../code/custom -I../code/custom/generated -DMETA_PASS -E 4coder_default_bindings.cpp -o 4coder_default_bindings.i
+```
+
+#### Stage 2: Metadata Generator Compilation
+```bash
+clang++ -I../code -I../code/custom -I../code/custom/generated -o metadata_generator 4coder_metadata_generator.cpp
+```
+
+#### Stage 3: Metadata Generation
+```bash
+./metadata_generator -R ../code/custom/generated/ 4coder_default_bindings.i
+```
+
+#### Stage 4: Final Custom Layer Compilation
+```bash
+clang++ -I../code -I../code/custom -I../code/custom/generated -shared -fPIC -o custom_4coder.so 4coder_default_bindings.cpp
+```
+
+### 4. Compiler-Specific Configurations
+
+#### Visual Studio (CL):
+```cpp
+#define CL_OPTS \
+"-W4 -wd4310 -wd4100 -wd4201 -wd4505 -wd4996 " \
+"-wd4127 -wd4510 -wd4512 -wd4610 -wd4390 " \
+"-wd4611 -wd4189 -WX -GR- -EHa- -nologo -FC"
+
+#define CL_LIBS_COMMON \
+"user32.lib winmm.lib gdi32.lib opengl32.lib comdlg32.lib userenv.lib "
+```
+
+#### GCC (Linux):
+```cpp
+#define GCC_OPTS \
+"-Wno-write-strings " \
+"-D_GNU_SOURCE -fPIC " \
+"-fno-threadsafe-statics -pthread " \
+"-Wno-unused-result " \
+"-std=c++11"
+
+#define GCC_LIBS_COMMON \
+"-lX11 -lpthread -lm -lrt " \
+"-lGL -ldl -lXfixes -lfreetype -lfontconfig"
+```
+
+#### Clang (macOS):
+```cpp
+#define CLANG_OPTS \
+"-Wno-write-strings -Wno-deprecated-declarations " \
+"-Wno-comment -Wno-switch -Wno-null-dereference " \
+"-Wno-tautological-compare -Wno-unused-result " \
+"-Wno-missing-declarations -Wno-nullability-completeness " \
+"-std=c++11 "
+
+#define CLANG_LIBS_COMMON \
+"-framework Cocoa -framework QuartzCore " \
+"-framework CoreServices " \
+"-framework OpenGL -framework IOKit -framework Metal -framework MetalKit "
+```
+
+### 5. Build Process Flow
+
+#### Development Build Flow:
+1. **Entry Script** (platform-specific) compiles `4ed_build.cpp`
+2. **Meta-Build Program** generates compiler commands based on platform/arch/flags
+3. **Custom Layer Build** (parallel process):
+   - Preprocesses source with meta macros
+   - Builds metadata generator
+   - Generates API bindings
+   - Compiles custom layer as shared library
+4. **Core Engine Build**:
+   - Compiles `4ed_app_target.cpp` as shared library (`4ed_app.so`)
+   - Compiles platform layer as main executable (`4ed`)
+5. **Resource Management**: Copies themes, fonts, configs to build directory
+
+#### Packaging Build Flow:
+1. **Standard Build** (as above)
+2. **Distribution Preparation**:
+   - Creates versioned directory structure
+   - Copies executables and shared libraries
+   - Includes source code (Super tier only)
+   - Copies documentation and resources
+3. **Archive Creation**: Creates ZIP files for distribution
+
+### 6. Key Build Outputs
+
+The build system produces:
+- **`4ed`**: Main editor executable (platform layer)
+- **`4ed_app.so`**: Core application logic (shared library)
+- **`custom_4coder.so`**: Custom layer with user commands (shared library)
+- **Resources**: Themes, fonts, configuration files
+- **Debug Symbols**: `.dSYM` directories (macOS), `.pdb` files (Windows)
+
+### 7. Build Complexity Analysis
+
+The current system is complex due to:
+
+1. **Multi-tier Architecture**: Three separate compilation units with interdependencies
+2. **Cross-platform Abstraction**: Different compilers, flags, and linking requirements
+3. **Dynamic API Generation**: Metadata generation for extensible command system
+4. **Packaging Requirements**: Multiple build tiers and distribution formats
+5. **Debug/Release Variants**: Different optimization and debugging configurations
+
+### 8. Simplification Opportunities
+
+The build system could be simplified by:
+
+1. **Unified Build Scripts**: Single bash script per platform instead of meta-build system
+2. **Direct Compiler Calls**: Eliminate C++ meta-build program in favor of direct clang/gcc invocations
+3. **Simplified Metadata**: Reduce or eliminate runtime metadata generation
+4. **Merged Compilation Units**: Combine core and custom layers into single executable
+5. **Standardized Toolchain**: Use consistent compiler (clang) across all platforms
+
+The current complexity stems from the editor's plugin architecture and cross-platform requirements, but many of these features could be maintained with a simpler build system that uses direct bash scripts calling clang with appropriate flags.

triage/build-system-simplification-plan.md

@@ -0,0 +1,396 @@
+# 4coder Build System Simplification Plan
+
+## Overview
+
+This document outlines a comprehensive plan to replace the current complex meta-build system with simple, direct bash scripts that call clang with appropriate flags. The plan maintains all essential functionality while eliminating unnecessary complexity.
+
+## Goals
+
+1. **Simplicity**: Replace meta-build C++ program with direct bash scripts
+2. **Maintainability**: Clear, readable build scripts with inline documentation
+3. **Standardization**: Use clang across all platforms for consistency
+4. **Functionality**: Preserve all essential build features and outputs
+
+## Current vs Target Architecture
+
+### Current System
+```
+Platform Script → Compile 4ed_build.cpp → Execute 4ed_build → Generate Commands → Build
+```
+
+### Target System
+```
+Platform Script → Direct Clang Calls → Build
+```
+
+## Phase 1: Foundation and Analysis (2-3 days)
+
+### TODO 1.1: Set Up New Build Directory Structure
+- [ ] Create `build_new/` directory for new build scripts
+- [ ] Create subdirectories:
+  - `build_new/scripts/` - Main build scripts
+  - `build_new/helpers/` - Helper scripts and utilities
+  - `build_new/config/` - Build configuration files
+  - `build_new/temp/` - Temporary files during build
+
+### TODO 1.2: Extract Essential Build Parameters
+- [ ] Analyze `4ed_build.cpp` to extract all compiler flags per platform
+- [ ] Document all linker flags and library dependencies
+- [ ] Create configuration file with all essential build parameters:
+  ```bash
+  # build_new/config/build-config.sh
+  # Platform-specific compiler flags, library paths, etc.
+  ```
+
+### TODO 1.3: Create Platform Detection Utility
+- [ ] Write `build_new/helpers/detect-platform.sh`:
+  ```bash
+  #!/bin/bash
+  # Returns: win32, linux, or macos
+  ```
+
+### TODO 1.4: Verify Clang Installation Requirements
+- [ ] Document clang version requirements for each platform
+- [ ] Create `build_new/helpers/check-toolchain.sh` to verify clang availability
+- [ ] Document any additional platform-specific setup needed
+
+## Phase 2: Core Build Script Development (4-5 days)
+
+### TODO 2.1: Create Master Build Script
+- [ ] Create `build_new/scripts/build.sh` as main entry point:
+  ```bash
+  #!/bin/bash
+  # Usage: ./build.sh [platform] [config] [arch]
+  # Examples: 
+  #   ./build.sh macos debug x64
+  #   ./build.sh linux release x64
+  #   ./build.sh win32 debug x64
+  ```
+
+### TODO 2.2: Implement Platform-Specific Build Functions
+- [ ] Create `build_new/scripts/build-macos.sh`:
+  ```bash
+  #!/bin/bash
+  # macOS-specific build logic using clang
+  # Handles Objective-C++ files, frameworks, etc.
+  ```
+- [ ] Create `build_new/scripts/build-linux.sh`:
+  ```bash
+  #!/bin/bash
+  # Linux-specific build logic using clang
+  # Handles X11, pthread, fontconfig, etc.
+  ```
+- [ ] Create `build_new/scripts/build-win32.sh`:
+  ```bash
+  #!/bin/bash
+  # Windows-specific build logic using clang
+  # Handles Windows APIs, resource compilation, etc.
+  ```
+
+### TODO 2.3: Implement Core Engine Build
+- [ ] Create `build_new/scripts/build-core.sh`:
+  ```bash
+  #!/bin/bash
+  # Builds 4ed_app.so (core application logic)
+  # Builds 4ed executable (platform layer)
+  ```
+- [ ] Extract and implement all essential compiler flags
+- [ ] Extract and implement all essential linker flags
+- [ ] Handle FreeType and other external libraries
+
+### TODO 2.4: Implement Custom Layer Build
+- [ ] Create `build_new/scripts/build-custom.sh`:
+  ```bash
+  #!/bin/bash
+  # Multi-stage custom layer build process
+  # 1. Preprocess with -DMETA_PASS
+  # 2. Build metadata generator
+  # 3. Generate metadata files
+  # 4. Compile custom layer shared library
+  ```
+- [ ] Preserve metadata generation functionality
+- [ ] Preserve API binding generation
+
+## Phase 3: Advanced Features (3-4 days)
+
+### TODO 3.1: Implement Build Configuration System
+- [ ] Create `build_new/config/flags-common.sh` - Common compiler flags
+- [ ] Create `build_new/config/flags-debug.sh` - Debug-specific flags
+- [ ] Create `build_new/config/flags-release.sh` - Release-specific flags
+- [ ] Create `build_new/config/libs-[platform].sh` - Platform-specific libraries
+
+### TODO 3.2: Implement Resource Management
+- [ ] Create `build_new/scripts/copy-resources.sh`:
+  ```bash
+  #!/bin/bash
+  # Copies themes, fonts, configs to build directory
+  # Handles platform-specific resource requirements
+  ```
+
+### TODO 3.3: Add Build Validation
+- [ ] Create `build_new/scripts/validate-build.sh`:
+  ```bash
+  #!/bin/bash
+  # Verifies all expected build outputs exist
+  # Checks library dependencies are resolved
+  # Validates executable functionality
+  ```
+
+### TODO 3.4: Implement Clean Build Support
+- [ ] Create `build_new/scripts/clean.sh`:
+  ```bash
+  #!/bin/bash
+  # Removes all build artifacts
+  # Supports selective cleaning (debug/release, platform-specific)
+  ```
+
+## Phase 4: Packaging and Distribution (2-3 days)
+
+### TODO 4.1: Create Packaging System
+- [ ] Create `build_new/scripts/package.sh`:
+  ```bash
+  #!/bin/bash
+  # Usage: ./package.sh [platform] [tier] [arch]
+  # Creates distribution archives
+  ```
+- [ ] Implement directory structure creation
+- [ ] Implement file copying for distribution
+- [ ] Implement archive creation (ZIP format)
+
+### TODO 4.2: Implement Version Management
+- [ ] Create `build_new/scripts/version.sh`:
+  ```bash
+  #!/bin/bash
+  # Extracts version from 4coder_version.h
+  # Generates version-specific naming
+  ```
+
+### TODO 4.3: Create Distribution Validation
+- [ ] Create `build_new/scripts/validate-package.sh`:
+  ```bash
+  #!/bin/bash
+  # Validates package contents
+  # Checks all required files are present
+  ```
+
+## Phase 5: Testing and Validation (3-4 days)
+
+### TODO 5.1: Create Test Suite
+- [ ] Create `build_new/tests/test-build.sh`:
+  ```bash
+  #!/bin/bash
+  # Comprehensive build testing
+  # Tests all platform/config combinations
+  ```
+- [ ] Test debug builds on all platforms
+- [ ] Test release builds on all platforms
+- [ ] Test packaging on all platforms
+
+### TODO 5.2: Performance Comparison
+- [ ] Create `build_new/tests/benchmark-build.sh`:
+  ```bash
+  #!/bin/bash
+  # Compares build times: old vs new system
+  # Measures clean build times
+  # Measures incremental build times
+  ```
+
+### TODO 5.3: Regression Testing
+- [ ] Verify all build outputs are identical to current system
+- [ ] Test custom layer loading functionality
+- [ ] Test metadata generation correctness
+- [ ] Test API binding generation correctness
+
+## Phase 6: Integration and Migration (2-3 days)
+
+### TODO 6.1: Update Project Files
+- [ ] Update `code/project.4coder` to use new build scripts
+- [ ] Update `code/custom/project.4coder` for custom layer builds
+- [ ] Update F-key bindings to use new build system
+
+### TODO 6.2: Create Migration Documentation
+- [ ] Document differences between old and new systems
+- [ ] Create migration guide for developers
+- [ ] Update CLAUDE.md with new build commands
+
+### TODO 6.3: Backup and Transition
+- [ ] Create backup of current build system
+- [ ] Implement gradual transition plan
+- [ ] Create rollback procedure if needed
+
+## Implementation Details
+
+### Script Structure Template
+
+Each build script should follow this structure:
+
+```bash
+#!/bin/bash
+set -e  # Exit on error
+
+# Script: build-[platform].sh
+# Purpose: Build 4coder for [platform] using clang
+# Usage: ./build-[platform].sh [debug|release] [x64|x86]
+
+# =============================================================================
+# Configuration
+# =============================================================================
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+BUILD_ROOT="$SCRIPT_DIR/../../build"
+CONFIG_DIR="$SCRIPT_DIR/../config"
+
+# Source configuration files
+source "$CONFIG_DIR/build-config.sh"
+source "$CONFIG_DIR/flags-common.sh"
+
+# =============================================================================
+# Platform-specific settings
+# =============================================================================
+
+setup_platform_vars() {
+    # Set platform-specific variables
+    # - Compiler flags
+    # - Library paths
+    # - Include paths
+    # - Linker flags
+}
+
+# =============================================================================
+# Build functions
+# =============================================================================
+
+build_core_engine() {
+    echo "Building core engine..."
+    
+    # Document each clang call with comments
+    # Example:
+    # Build 4ed_app.so (core application logic)
+    clang++ \
+        -shared \
+        -std=c++11 \
+        -I"$CODE_DIR" \
+        -I"$FOREIGN_DIR/freetype2" \
+        -DFTECH_64_BIT \
+        -DFRED_SUPER \
+        -O2 \
+        -o "$BUILD_DIR/4ed_app.so" \
+        "$CODE_DIR/4ed_app_target.cpp" \
+        "$FOREIGN_DIR/x64/libfreetype-mac.a"
+}
+
+build_custom_layer() {
+    echo "Building custom layer..."
+    
+    # Stage 1: Preprocess with meta macros
+    clang++ \
+        -I"$CODE_DIR" \
+        -I"$CUSTOM_DIR" \
+        -DMETA_PASS \
+        -E \
+        "$CUSTOM_DIR/4coder_default_bindings.cpp" \
+        -o "$BUILD_DIR/4coder_default_bindings.i"
+    
+    # Stage 2: Build metadata generator
+    clang++ \
+        -std=c++11 \
+        -I"$CODE_DIR" \
+        -I"$CUSTOM_DIR" \
+        -o "$BUILD_DIR/metadata_generator" \
+        "$CUSTOM_DIR/4coder_metadata_generator.cpp"
+    
+    # Stage 3: Generate metadata
+    "$BUILD_DIR/metadata_generator" \
+        -R "$CUSTOM_DIR/generated/" \
+        "$BUILD_DIR/4coder_default_bindings.i"
+    
+    # Stage 4: Compile custom layer
+    clang++ \
+        -shared \
+        -std=c++11 \
+        -I"$CODE_DIR" \
+        -I"$CUSTOM_DIR" \
+        -I"$CUSTOM_DIR/generated" \
+        -DFTECH_64_BIT \
+        -DFRED_SUPER \
+        -O2 \
+        -o "$BUILD_DIR/custom_4coder.so" \
+        "$CUSTOM_DIR/4coder_default_bindings.cpp"
+}
+
+# =============================================================================
+# Main execution
+# =============================================================================
+
+main() {
+    setup_platform_vars
+    create_build_dirs
+    build_core_engine
+    build_custom_layer
+    copy_resources
+    validate_build
+    
+    echo "Build completed successfully!"
+}
+
+main "$@"
+```
+
+### Error Handling Strategy
+
+Each script should include:
+- `set -e` for exit on error
+- Validation of input parameters
+- Checking for required tools and dependencies
+- Clear error messages with suggested fixes
+- Cleanup of temporary files on failure
+
+### Documentation Standards
+
+Each script should include:
+- Purpose and usage in header comments
+- Inline documentation for each major step
+- Clear variable names and consistent formatting
+- Examples of usage in comments
+
+## Success Criteria
+
+The new build system will be considered successful when:
+
+1. **Functionality**: All current build outputs are preserved
+2. **Simplicity**: No C++ meta-build program required
+3. **Maintainability**: Clear, readable bash scripts
+4. **Performance**: Build times equal or better than current system
+5. **Reliability**: No regression in build success rates
+6. **Documentation**: Clear usage instructions and examples
+
+## Risks and Mitigation
+
+### Risk 1: Platform-specific Compatibility Issues
+- **Mitigation**: Extensive testing on all target platforms
+- **Fallback**: Maintain current system until new system is fully validated
+
+### Risk 2: Missing Build Features
+- **Mitigation**: Comprehensive analysis of current system before implementation
+- **Fallback**: Gradual migration with feature parity validation
+
+### Risk 3: Performance Regression
+- **Mitigation**: Benchmark testing and optimization
+- **Fallback**: Performance profiling and targeted improvements
+
+### Risk 4: Developer Disruption
+- **Mitigation**: Clear migration documentation and parallel system support
+- **Fallback**: Quick rollback procedure if issues arise
+
+## Timeline Summary
+
+- **Phase 1**: 2-3 days - Foundation and analysis
+- **Phase 2**: 4-5 days - Core build script development
+- **Phase 3**: 3-4 days - Advanced features
+- **Phase 4**: 2-3 days - Packaging and distribution
+- **Phase 5**: 3-4 days - Testing and validation
+- **Phase 6**: 2-3 days - Integration and migration
+
+**Total Estimated Time**: 16-22 days
+
+This plan provides a comprehensive roadmap for replacing the complex meta-build system with simple, maintainable bash scripts while preserving all essential functionality and ensuring a smooth transition for developers.