# Changelog

v0.75 (11-Dec-2014)
Improvements

- Large optimization for highly recursive dependency graphs
- Progress calculation optimized
- Progress calculation accuracy greatly improved

v0.74 (1-Dec-2014)
Note

- Database version changed : Clean build will occur
- Various internal changes to prepare for Linux & OSX

New

- CopyDir function - copy contents of one or more directories

Improvements

- Add explicit check and error msg for missing DLLs on remote machine
- Alias can refer directly to files not built by FASTBuild

Fixes

- Massive speedup to directory listing performance when dealing with > 4096 files
- Copy() dest timestamp is correctly set when using PreBuildDependencies
- #once include directive not being respected in some cases
- Fix included bff being considered always dirty in some cases
- VCXProjectNodes can be referenced as files (allows copying)
- Various cases where empty strings was not correctly reported as error 1004
- Empty Alias() .Targets is reported as an error instead of silently doing nothing

v0.73 (11-Nov-2014)
Note

- Database version changed : Clean build will occur
- Various internal changes to prepare for Linux & OSX

New

- Link steps (DLL/Exe) have new options for post-link in-place stamping (.LinkerStampExe and .LinkerStampExeArgs)

Fixes

- Preserve case of files discovered during compilation
- Fix BFF parser crash caused by unterminated strings, arrays or structs under some circumstances (now emit Error 1002)
- Ensure UnityInputExcludedFiles works correctly with full paths
- Ensure directories for VS Project generation are created before use
- Clang detection: *clang[.exe] and *clang-cl[.exe] detected in addition to existing *clang++[.exe]

v0.72 (28-Sep-2014)
Note

- Database version changed : Clean build will occur

New

- Support for Wii toolchain (CodeWarrior)
- Support for WiiU toolchain (GreenHills)
- Support for CUDA (nvcc.exe) compiler

Improvements

- Synchronized compiler files are locked by FBuildWorker.exe to prevent deletion on remote machine
- Project generation includes CUDA *.cu files by default

Fixes

- UnityInputExcludedFiles matches only full filenames instead of just matching the end of file names
- UnityInputExcludedFiles are canonicalized so double slashes no longer cause failure to exclude files
- Gracefully handle compiler being terminated remotely due to shutdown of PC

v0.71 (19-Jul-2014)
Note

- DB version has been increased; clean build will result after upgrading
- Distributed worker protocol has changed; update workers

New

- "-showcmds" option shows commands being executed during compilation (far less verbose than verbose)
- "-cpus=n" option allows number of cpus to be controlled from the command line
- "-mode=x" option allows worker mode to be controlled from the command line
- "-wait" command line options to allow queueing of multiple builds
- Add DeoptimizeWritableFiles option (use deoptimized settings for writable files)

Improvements

- Prevent Executables/DLLs re-linking when a DLL they depend on changes, but the interface stays the same
- Incremental link files (.ilk) deleted before linking when doing a -clean build
- PDB files deleted before linking (if non-incremental) to avoid pdb bloat and link slowdown
- Automatically strip /MP argument for MSVC compilers (FASTBuild controlls the parallelization)

Fixes

- Differentiate remote "system" failures from compilation failures
  - a remote failure will cause the worker to be blacklisted and the task retried on another worker
  - remote failure on 3 different workers will fail as before (indicates a configuration problem)
- Error path fixup now handles "remark:" and "fatal error:" style messages
- Fix path fixup for SNC compiler output (errors, warnings etc)
- PS4 linker uses response files to avoid 32KiB cmd line limit
- PS4 linker tracks library dependencies passed directly on compiler command line
- FBuildWorker.exe correctly exits on WM_CLOSE notifications (taskkill for example) instead of minimizing
- PreBuildDependencies are correctly processed for -summary and -report

v0.70 (21-Jun-2014)
Note

- DB version has been increased; clean build will result after upgrading
- Cache format has changed; existing cache data will be ignored
- Distributed worker protocol has changed; update workers

New

- Distributed compilation is no longer in Beta.

Improvements

- Cache size & speed optimizations
  - Cache compression switched to LZ4
  - 20-30% reduced output size (less space used on cache, less network transfer)
  - 20-30% faster compression (less time spent on local machine when storing to cache)
  - 600% faster decompression speed (when decompressing on local machine after retrieval from cache)
- FBuildWorker can use tools synchronized from a previous run to reduce network IO
- Distributed compilation speed optimizations
  - Replace distributed compilation compression with LZ4
  - 50% reduced transfer size (half the data sent to remote machine, half the memory usage locally when pending)
  - 230% faster compression speed (less time spent compressing before sending)
  - 700% faster decompression speed (7x less work to decompress on remote machine)
- Use 64bit MurMur3 hash for faster/better cache key generation
  - 300% faster hash of preprocessed cache input
  - Less chance of collisions vs CRC32
  - Using 64bits MurMur3 for further reduced collision potential
- Optimize include parsing for all compilers
- Support /7i and /ZI debug format for distribution with MSVC compiler

Fixes

- Handle tool synchronization failure properly to avoid leaked jobs
- Fix remote worker over-requesting jobs when not using full cpu allocation (was causing reduced scheduling efficiency)
- Includes no longer implictly push an addition {} scope (this was preventing includes being useful in many cases)
- -config args are correctly passed to sub-process when in -vs or -wrapper mode
- Fix race condition preventing copy in FBuildWorker auto-restart, causing update to fail

v0.62 (28-Apr-2014)
Note

- DB version has changed: full build will occur
- Wrapper Mode (-wrapper) is now active as part of Visual Studio mode (-vs)

New

- FBuildWorker will auto-restart when updated (i.e. new version sync'd from revision control)

Improvements

- Allow ObjectList, Library and CSAssembly to take multiple exclusion paths
- Add "DeploymentType" and "DeploymentFiles" to vcxproj generation for Xbox360
- .m and .mm files are included in vcxproj generation by default
- Multiple targets are built in parallel
- Cost of progress calculation is massively reduced
- Remotely compiled tasks are written to the cache on return
- -summary shows total remote CPU time
- Reduce FBuildWorker CPU usage when worker is idle

Fixes

- Deoptimized compilation settings were being ignored if using distributed compilation
- Files using /FI (Force Include) with MSVC compiler compile correctly when using the cache or distribution
- Fix -summary time output when building multiple targets
- Consider PreBuildDependencies correctly in progress estimation
- Fix report/chart colours not working in -report due to bad html
- Fix bad report generation when using ObjectList
- Fix collapsable report sections in IE and Firefox
- Correct -summary build count for remotely compiled objects
- Locally build distributable tasks can be written to the cache
- Correct use of reponse files with orbis-ar.exe which requires slashes to be escaped in file

v0.61 (29-Mar-2014)
Note

- DB version has been increased; clean build will result after upgrading
- Cache format has changed; existing cache data will be ignored
- Distributed worker protocol has changed; update workers

New

- Cache Plugin Architecture - allows cache IO functionality to be provided by a plugin dll

Improvements

- Distributed Compilation

- Increase distributed connection limit to 15
- Minimization state of worker is persisted between runs
- Worker default CPU allocation is 50%
- Add icon the FBuildWorker.exe
- Memory usage for distributed compilation is massively reduced
- Distributed compilation will race remaining remote jobs locally, when there is no other work to do

- Improve validation of .LinkerOutput argument (ensure a file and not a path is provided)
- Safely handle empty libraries passed to linker .Libraries
- Check for and handle truncated data being returned from the cache

Fixes

- Distributed Compilation

- Fix crash in worker when compilation error contains %
- Fix job leak causing hang when synchronizing the same toolchain for 2 clients at the same time
- Prevent some unsafe access to management structures on Worker
- Fix potentially long shutdown when a remote worker is unresponsive
- Prevent potential hangs in networking
- Fix memory leak in worker if disconnected during tool synchronization

- Fix crash when doing multiple rapid build/cancel cyles with -wrapper mode
- Use the compiler toolchain hash in the cache key
- Fix inability to detect fully pathed libs passed to .LinkerOptions
- Detect MSVC linker correctly when provided as file only (i.e. relying on system path for discovery)
- Fix path canonicalization when a relative path begins with a slash
- Fix crash if cache path is empty, or a UNC path only to a machine
- A failure in one of multiple targets on the command line no longer causes all subsequent tasks to fail
- Librarian step uses response files to avoid 32KiB command line length limit

v0.60 (10-Mar-2014)
Note

- DB version has been increased; clean build will result after upgrading

New

- [BETA] Distributed compilation support
- Deoptimization of writable files containing FASTBUILD_DEOPTIMIZE_OBJECT token

Improvements

- Project Generation

- Generated projects can directly launch/debug PS3/PS4
- Allow options to be set per-configuration
- Many new project generation options exposed
- Supports multiple base paths

- Caching

- Show cache hits in standard output
- Caching automatically disabled for files isolated from unity
- Cache mode can be controlled via FASTBUILD_CACHE_MODE environment variable ('r', 'w' or 'rw')
- Allow cache path to be set via an environment variable "FASTBUILD_CACHE_PATH"

- FASTBuild runs at below normal priority for improved machine responsiveness
- When MS linker crashes and we retry, supress the error to prevent visual studio from failing
- Exec with -verbose prints in a cut and pasteble format to assist config debugging
- Saving of the dependency graph is performed atomically to reduce chance of corruption if fbuild.exe is terminated during DB serialization
- [BETA] Add "-wrapper" command line argument for Process Isolation. Allows graceful shutdown when terminated from Visual Studio.
- Add .PreBuildDependencies to Copy() and Exec()
- Add '-fixuperrorpaths' command line to reformat GCC/SNC/Clang errors to Visual Studio format
- Prevent two objects being built to the same obj
- Enabled Unity generation on multiple threads
- PrecompiledHeader is used when locally building a distributable job, or if cache is in read-only mode

Fixes

- Prevent #pragma messages being incorrectly parsed as includes in some cases with GCC/SNC/Clang
- Fix -I arguments not being removed from Clang when using preprocessed output
- Precompiled headers with Clang are not prevented from working usefully
- Don't try to use precompiled headers in link step with GCC/SNC/Clang
- Project generation LocalDebuggerWorkingDirectory written to correct part of project
- Fix missing carriage return in output with --help
- Exec with -verbose prints args after substitutions (%1 etc)
- Handle UNC cache path correctly
- Format FASTBuild errors so that column is interpretted by VisualStudio correctly
- Fix performance degradation when failure output is massive (>1MiB) - (e.g. link failure)
- Correctly return error when multiple targets are built and one fails
- Replace pattern matching of FindNextFile with our own (so *.cpp doesn't return files like *.cpp~)
- Fix items not under project path not being double clickable
- Library() deleted libraries before creation to ensure old symbols don't remain in library
- Prevent bff included with ../ being considered dirty every time

v0.58 (23-Jan-2014)
Note

- DB version has been increased; clean build will result after upgrading
- BFF includes are now relative to includer (no "system" includes)

New

- Clang & PS4 support
- Add #once pragma which allows BFF files to be included multiple times, but parsed only once
- MSVC linker crashes (LNK1000) will be retried once as work around for bug in x64 incremental linker

Improvements

- VCXProject creates projects which are compatible with VS2010 as well as 2012/2013
- VCXProject can specify debugger args (working dir, startup exe etc)
- Path canonicalization handles \.\ and \..\ for better consistency in target names
- Failure find include shows full path of where the include was expected to be found
- Greatly improve performance of VS project generation when there are many folders
- Dependencies passed directly to linker (instead of .Libraries) and have their dependencies tracked
- Copy step shows source and dest for copy in tty
- -verbose mode will specify the reason for building any nodes
- Exec function supports build time substitutions %1 (input) and %2 (output)
- /IMPLIB supported for MSVC linker. Import lib specified will be used by dependent dlls.
- Show in output when DB version has changed and rebuild will occur
- Show when BFF files are modified and config re-parsing will occur
- CachePath of wrong type emits an error instead of silently being ignored
- Various system error messages are now formatted to appear in Visual Studio error window
- Improve GCC detection - recognize g++ as well as gcc
- Repeat name of object when compilation fails so task is clear

Fixes

- Caching can be safely used on objects using precompiled headers with MSVC, avoiding missing debug info.
- CLR code is not distributed/cached (compiler bug prevents safe use of the preprocessor)
- Fix temp file writing failure detection (ensure whole file is written)
- Compiler and Linker use response files to avoid Windows 32KiB args limit
- Detect cyclic BFF include dependencies and report new Error 1035 instead of crashing with stack overflow
- Correctly detect changes to BFF files with the same name when one is bff relative and one is in the system directory
- Fix duplicate folders in generated VS project, causing project size bloat
- Correctly handle '-o FileName' as well as '-oFileName' arguments with GCC/SNC compiler when manipulating args
- Fix error in command line arguments to SNC when using cache
- Fix bad file extension for preprocessed temp file with SNC compiler
- Disable cache when using /ZW (consume WinRT) with MSVC compiler due to incompatibility
- Fix cache sensitivity on working dir drive letter causing cache misses
- Correct reversed types in Error 1027 with String + ArrayOfStrings
- Fix formatting so first error with GCC/SNC is always double-clickable in Visual Studio

v0.57 (8-Dec-2013)
Note

- DB version has been increased; clean build will result after upgrading

New

- VisualStudio project generation
- New .LinkerLinkObjects option on Executable() & DLL() allows control over linking objects instead of libs.
- New .CompilerForceUsing option on Library() and ObjectList(), with %4 build-time substitution for use with MSVC /FU option.
- New .LinkerAssemblyResources option on Executable() & DLL(), with %3 build-time substitution for use with MSVC /ASSEMBLYRESOURCE option.
- New .PreBuildDependencies option for Library(), allows forced execution of an external process before a Library/ObjectList is built to better support extenal code generation.
- New ObjectList() function allows building a list of items without linking into a library, to support buiding objects or resources without making libraries.
- .LibrarianAdditionalInputs option on Library() to allow merging of objects, libraries or resources built externally or with Exec() or ObjectList().

Improvements

- Add '.UnityInputFiles' option to allow an explicit file list to be provided to a Unity node
- Support various assignment/concatenation oprerations with ArrayOfStrings and ArrayOfStructs
- For libraries with mixed Unity and Non-Unity nodes, favour Unity nodes for compilation
- Prioritize linking over compiling to reduce DLL bottlenecks
- Job queue manipulation overhead is reduced
- DLL & Executable support referencing a library or dll via a copy (to ensure the copy happens at build time)
- Copy() supports multiple inputs if an output path is specified
- Unity files emit #pragma messages to clearly show file being compiled in case of failure
- .CompilerOutputExtension can be specified on Library() to control the extension for built objects
- FASTBuild emits an error and exits if another instance is already running
- Allow use of %3 Compiler build-time expansion for PCH object in object compilation as well as precompiled header compilation

Fixes

- Escaping correctly works for all characters, not just "
- -summary option shows cache hit time for cache hit nodes, instead of last known build time
- Prevent a race where some additional jobs could be started after a failed task
- Fix incorrect error message when attempting to concatenate Struct to ArrayOfStrings
- Fix crash when self-assigning variables (i.e. .MyString = .MyString)
- Assignment of structs was functioning as a concatenation (was keeping old members when it shouldn't)
- Handle quoted build-time substitutions correctly to support spaces in paths (i.e. "%1" becomes "A" "B" "C" (instead of "A B C")
- Unity uses "" style includes (instead of <>) to satisfy compilers like Clang
- Fix bug causing massive slowdown generating large reports (-report)
- Fix output to stdout being truncated to 8KiB in some cases when using -verbose
- Copy can reference an alias as a source, including those aliasing multiple inputs
- Correct bad formatting of Exec node in -verbose mode
- Executable & DLL linking can accept an input built by Executable()
- Fix failure to extract includes when include as on last line with no carriage return

v0.56 (10-Nov-2013)
Note

- DB version has been increased; clean build will result after upgrading

New

- New -report command line argument which outputs a detailed report of the build, including:
  - Processing time by item.
  - Processing time by library.
  - Processing time by type (obj, exe, dll etc).
  - Cache statistics.
  - Include file usage.
- Add 'UnityInputIsolateWritableFiles' to unity generation. Allows files which are "open for edit" in revision control to be built individually to speed up iterative compile times.

Improvements

- Tests can reference execuable via alias
- -verbose output for Objects is improved: Timing is more accurate and number of includes found is displayed.
- Include processing optimizations
- Dependency graph traversal optimizations
- Progress estimation optimizations
- Large database load/save time is greatly reduced
- Database size is greatly reduced
- Total build time includes some previously excluded overheads
- Job queue contention optimization
- Small optimization to Unity generation

Fixes

- Avoid using preprocessor with MSVC compiler unnecessarily
- Fix C# dependencies not being tracked properly when using -clean
- Fix C# dependencies not being tracked correctly for multiple explicitly listed .cs files
- GCC includes not being correcly detected
- MSVC includes being missed on some rare occasions
- Fix poor unity splitting logic resulting in bad distribution and occassional empty unity files
- Cache stats in -summary were not being displayed when in "writeonly" mode
- Cache commit to existing entry no longer results in removal of entry
- Allow concatenation of a String variable to an ArrayOfStrings variable instead of generating error 1034
- Copy step outputs a line to the tty during build, consistent with other tasks
- Copy outputs detailed information when -verbose is used
- Copy outputs a failure message when copy fails
- Fix progress calculation reaching 100% before completion
- Fix small memory leak when generating unity files
- Handle some highly unlikely cases of failing to set file time for cache retrieval
- Empty arrays containing whitespace or comments are now handled correctly

v0.55 (5-Oct-2013)
Note

- DB version has been increased; clean build will result after upgrading

Improvements

- Language Syntax Enhancements

- A new variable of type Struct can be defined, grouping other variables
- Structures can be kept in Arrays and Arrays can be iterated over
- A new Using function takes all the variables in a Struct and pushes them into the current scope
- Add Print function to assist debugging configurations
- Allow 0 size ForEach loops
- Allow String to be concantenated to ArrayOfStrings

- Build Functionality Enhancements

- Add SNC compiler detection
- Add Exec() function - calls an arbitrary executable to create a file
- Test() function can use existing executable, as well as one built by FASTBuild

- Static libs and DLLs automatically use precompiled header obj files
- Errors are output contiguously, improving readability when there are multiple failed nodes
- Improve thread number output when compiling with > 9 threads
- Unity 'InputUnityExludePath' supports multiple exlude paths
- More detailed errors when failing to write temp files
- Linker failure message is imrproved (specifies Dll or Exe, rather than just "Linker")
- Add wesbite and copyright info to "-version" output
- Show total time taken by node type in -summary output
- Various -summary formatting enhancements
- Small performance improvement for processing very large objects

Fixes

- DLL Fixes

- DLLs comiled with MSVC correctly link export symbols
- When linking a DLL, "DLL:" is shown in output instead of "Exe:"
- Linking a DLL with another DLL deduces the import library automatically
- Ensure DLLs are not rebuilt unnecessarily

- GCC/SNC/PS3 Fixes

- Fix broken PS3/SNC compilation (was reading stderr instead of stdout)
- Standardize GCC and MSVC compiler detection (allow with and without .exe extension)
- Fix detection of GCC compiler in PS3 SDK
- Fix detection of C vs C++ for GCC/SNC when using the cache

- Correct errors not being double-clickable in VisualStudio in some situations due to thread numbering
- Ensure active job count doesn't underflow when performing local jobs
- Correct formatting of cache path to stdout in -verbose mode
- Correct error message when attempting to append String to incompatible type
- Don't spin @ 0% when failing the build
- Save completion of items which are succesful even after the build as a whole has failed
- -summary formatting is no longer reliant on tab width

v0.54 (3-Jun-2013)
Note

- DB version has been increased; clean build will result after upgrading

Improvements

- Allow {} to be used alone to create a local scope (like a function)
- Config files (bff) can now include other config files, e.g. #include "file.bff"
- Direct variable assignment/concatenation now supported for variables (.X = .Y)

Fixes

- Unity generation parameter UnityInputExcludedFiles is no longer sensitive to slash type used
- Stop tasks sometimes being started after build is aborted with Ctrl+C
- Fix typographical error in command line usage output
- Fix hang at startup if database file (.fdb) has been corrupted
- Fix corrupt "Unknown argument" error message when passed bad cmd line arguments

v0.53 (24-Feb-2013)
Note

- The cache is invalidated due to path/version changes (no old entries will be used)

Improvements

- Cache performance and space usage improvements

- Cache directory hierarchy is simplified to reduce IO
- Cache contents are compressed, saving ~50% of disk space
- Compression yields speedups on IO bound (local or network) builds

- Handle transient file opening failures due to other applications opening our files (Antivirus etc)
- Add "Copy" function which can copy a file during the build
- Library and CSAssembly 'CompilerInputPath' can now accept a list of paths (or a single path as before)
- Optimize node lookups to reduce CPU usage
- Hide distributed task counts on progress bar when not in distributed mode

Fixes

- Correct misleading error when unexpected character follows a variable operator (was displaying #1016, now displays new #1017)
- Correctly handle comments in the middle of variable assignments (e.g. ".Var // comment" or ".Var = // comment")
- Comments are correctly parsed inside {} scopes when starting at column 0
- Add missing type safety checks on various CSAssembly and Library properties
- Unity files are forced to be regenerated when "-clean" option is used
- Build no longer gets stuck upon cache hit when running with "-j0"
- Temp directory now correctly initialized when running with "-j0"

v0.52 (18-Feb-2013)
Note

- The build database (fdb) version has changed, a clean build will forced on first use of this version

Improvements

- Unity 'UnityInputPath' can now accept a list of paths (or a single path as before)

Fixes

- Fix crash emitting a large errors to the TTY (>64KB)
- Detect non-unique Unity, Executable and DLL nodes (emit Error #1100)
- Handle Unity output paths that are not \ terminated

v0.51 (17-Feb-2013)
New

- Config file now supports // style line comments as well as ; style
- Add checks and errors (#1302, #1303) for bad precompiled header setup for cl.exe
- Config markup helpers for VisualStudio and NotePad++

Improvements

- Improve precision of timer used to track task durations

Fixes

- All object and pch files are deleted before compilation to ensure rebuild/no old files left behind
- Fix bad string output in "Unknown build target" message in x64
- Compiler executable (MSVC & GCC) detection is no longer case sensitive
- Removed detection of lower-case /z7 is as using Z7 debug format for cl.exe. (Z7 debug format arg is uppercase, and cl.exe compiler options are case sensitive)

v0.50 (12-Feb-2013)
The first public release of FASTBuild, with stable support for:

- Optimal multi-threaded compilation
- Build caching (network and/or local)
- Unity/Blob generation
- Unit Test Integration