Tiny Thor (made with Monkey X)
Develop on Windows or OSX and deploy easily
Crypt of the Necrodancer (made with Monkey X)
New Star Soccer - BAFTA Winner 2013!!! (made with Monkey X )
Ted - The code editor of Cerberus X
Avalon Legends (made with Monkey X)
Race Time (made with Monkey X)

WIP What I was working on before Xmas - A Cerberus-X Preview

dawlane May 30, 2018

  1. MikeHart

    MikeHart Administrator Staff Member Administrator Moderator

    Messages:
    1,231
    Likes Received:
    300
    Trophy Points:
    83
    Ok, then I will wait to test the buildscripts further till your new preview is ready. I had asked because depending on which variant I called the script, a tool didn't build. One time shared trans, then trans, or launcher. Ted was never build, but from within QT I can build it just fine. I will wait till your preview and report back.

    Local repository == the top folder you run the package option in?

    Powershell scripts is completely new to me and definitely something I need to learn.
     
  2. dawlane

    dawlane Active Member CX Code Contributor

    Messages:
    385
    Likes Received:
    161
    Trophy Points:
    43
    Yes the root directory of the locally cloned git repository that would be the main work copy of cerberus.
    When I build Cerberus, I open a shell, change directory into cerberus/src and execute the rebuild all scripts from there.

    The idea is that you have a full cloned git repository as your work copy. Here you can test out anything git related. Note you should make a backup of the local work repository before doing any merge or rebase in case things go wrong. Git is not entirely infallible:rolleyes:.

    You can build all the tools within the local work copy of the git repository. Depending on the tools use to edit any source files, the line endings could be in either crlf or lf. The --package/-p just copies the files as they are from the local work git repository. The final result is a directory name cerbdeploy/build that is created in the users Documents directory where it can be check that the script is copying correctly. The last thing you would want is to copy over is either system specific files or a restricted licence library that you may have missed with .gitignore.

    When the --git/-g option is passed it first makes a full clone of the local working repository into Documents/cerbclone fixing any line ending issues outlined in the .gitattribute file and the global git configuration settings. I have Git for Windows installed and use the fully cross platform GitKraken, it's free for non commercial use. If you are do commercial and none commercial, its best to use either the pro version, or go with a git client tool that doesn't place such restrictions, as long as that client installs git for use via the command line.

    Special note here: git cannot process non UTF-8 encode files that you tend to find in the Windows world. Special care must be taken with xml files where the UTF in the file's xml header can be different to the actual file encoding (discovered that with the Window Phone target and resource files). It's best to set them as binary, eol=lf or at a push, the actual encoding may work. You should check again that the files are correct before actual deployment. Git was originally a Linux tool and ported over to the other operating systems.

    The script then executes the freshly cloned rebuildall script with parameters that were passed, minus the --git option. It then takes over building within the new location. Once the build is done, it copies over the new built files (via the --packag/-p option) and compresses it ready for distribution. You can then check the local work repository and remove any temporary commits before pushing to the remote server.

    EDIT:
    The deploy package using git requires that it's a valid git repository. It will not work from a pre-built distributed package.

    Edit2: One thing that needs a small change is the rebuildall.sh script setting files permissions too early.

    It's still new to me. I wouldn't mind getting my hands on Powershell Studio to make working with Powershell scripts easier. But it ain't frigging cheap. There is suppose to be a free community version, but I think that you have to create an account to get it.

    My preferred tools are:
    Those mentioned above (apart from Powershell Studio).
    Visual Studio Code. It's cross platform, has a limited git client functionality that comes in handy when you don't want to mess around with your git client of choice. Has a built in terminal and limited script debugging depending on the platform is't being run on. It also supports the changing of line endings.
     
    Last edited: Oct 19, 2018
  3. Phil7

    Phil7 Active Member

    Messages:
    143
    Likes Received:
    36
    Trophy Points:
    28
    I just want to give you a thumbs up. I was not able to use JungleIde to compile for desktop with the official version, but with yours it works again. In between time I wrote with Jungle and compiled with ted ... not to much fun.
     
  4. dawlane

    dawlane Active Member CX Code Contributor

    Messages:
    385
    Likes Received:
    161
    Trophy Points:
    43
    Well I was hoping to have some more previews up and running. But getting things to work in Windows that way that I want is causing a bit of a headache.

    The idea is to have a new compiler detection module for transcc to use, and as always, it's Windows and it's tool chains that are causing a me a headache with having to check if MinGW is to be used or Visual Studio.

    So for the time being, I'm will posting a link to the compiler detection code for people to compile and test. It's ugly, but better than doing a hack of transcc just to detect the all the versions of MinGW. Extract and place the dawlane directory into modules_ext.

    It may break in the future.
     
  5. MikeHart

    MikeHart Administrator Staff Member Administrator Moderator

    Messages:
    1,231
    Likes Received:
    300
    Trophy Points:
    83
    Sounds like it is getting over engineered and to complicated.

    If it is complicated for you, how will it be for the average Joe?
     
  6. dawlane

    dawlane Active Member CX Code Contributor

    Messages:
    385
    Likes Received:
    161
    Trophy Points:
    43
    It's trying to integrate the compiler module into transcc in clean way. I'm trying to think of ways to internally simplify stdcpp and glfw target code to make it more manageable to work with. The current version on the experimental branch is more of a hack of the original code, especially the stdcpp target and there's a bit of code duplication that would be better of in the parent class.

    For the average Joe. They wouldn't need to have to set path's in config.winnt.txt for either MinGW or MSBuild, as the compiler module would try to get any installed compilers from known locations. But the editing of MinGW or MSBuild has to remain an option to set the compiler.
    Once I've figured it out, it shouldn't as complicated as having to write code for the translator. It's just wrapping my head round the best way to make sure that the right compiler tool chain is being used at the right time.
     
  7. MikeHart

    MikeHart Administrator Staff Member Administrator Moderator

    Messages:
    1,231
    Likes Received:
    300
    Trophy Points:
    83
    So what happens when you have both compilers installed?
     
  8. dawlane

    dawlane Active Member CX Code Contributor

    Messages:
    385
    Likes Received:
    161
    Trophy Points:
    43
    If both are installed, and no paths are set. Then the detection module will scan known locations for both MinGW and Visual Studio. If both are installed; then both will be stored into a stack. The default will always to try to use MinGW. If MinGW isn't found; then it will try to use Visual Studio.

    I think I've got it sorted and working the way that I want, so either later today or tomorrow I will have a Windows test version uploaded.
    It's not easy trying to debug transcc with transcc when messing around with the basic core build system. It can drive you :mad:.

    The only decision I have to make is whether to leave the common routines for glfw and stdcpp as functions in the helper files, or place them into the builder files as methods. Using the routines as functions means that the class reference has to be passed as a function parameter, but leaves the builder class as is with only a few extra variable fields.

    Here's basically how it should all work:

    The Detection module:
    Other than having a hundreds of check sums for binaries. The easiest solution is to try and execute the compiler and get any version information from the resulting output that can be scanned for meaningful information.

    By passing search paths, you can check for valid executable files for MSBUILD, GCC, XCODEBUILD and CLANG (Mac OS).
    If there are multiple paths passed and each one is to a valid compiler executable, then each one will be stored. Duplicates are not stored. So if two path were given and the executable is the same version; then only the first one found is stored.

    MinGW: user paths, default locations C\: and a Environment variable named MINGW are checked.

    Visual Studio 2015/2017: user paths, C:\Program Files\MSBuild, C:\Program Files\Microsoft Visual Studio\...etc
    Note: For Visual Studio 2017 update 2 Microsoft introduced a tool to locate Visual Studio installations. This is checked first for Visual Studio 2017 update 2+ and ignores any paths passed.​

    Linux and Mac OS don't need serious detection with them basically being already in the environment search path.

    transcc:
    The MINGW_PATH and MSBUILD_PATH are used to over-ride auto detection and should return any compilers at the beginning of the compiler list. There is no need to add msbuild.exe to MSBUILD_PATH as the executable would be part of the compiler data.

    Two new config.winn.txt variables are used for Visual Studio. VS_DEFAULT and VS_VERSIONS. The latter is a semi colon separated string with Visual Studio year versions, while the former is what the current version Cerberus is to use as a default.

    Three new common functions have been for glfw and stdcpp.
    1. SelectVSTUDIO: Use to select which Visual Studio version to use based on GLFW_VSTUDIO_VERSION and CC_VSTUDIO_VERSION.
    2. SetWindowsDesktopCompiler: Used to select either MinGW or Visual Studio based in the detected compilers and status GLFW_USE_MINGW and CC_USE_MING. If only one compiler is found, the preprocessor directives are ignored.
    3. CheckMSize: Used to select the binary architecture to output based on the values in the MSIZE preprocessor directive type.
    sharedtrans also uses the compiler detection module, as it was need to implement caching of library files. So the first run searches and caches, while the next run loads the cache file, checks that the right library is to be copied. This is doubly so for MinGW run-times.
     
    MikeHart likes this.
  9. dawlane

    dawlane Active Member CX Code Contributor

    Messages:
    385
    Likes Received:
    161
    Trophy Points:
    43
    New one for testing.

    I'm only putting the Windows version up for now as there is an odd bug where some errors are not being trapped in debug mode and I'm not sure what's going on. Try writing some test programs that throw index out of range and access violations. It seems to affect compiling with MINGW, but I have seen this issues with Visual Studio in C++Tool builds. I remember that there was an issue in MonkeyX where certain exceptions were not being caught in C++Tool targets. I get different results with Jungle and Ted, so I'm suspecting that The problem may be to do with piping stdout/stderr with one of the functions I've added, but I don't fathom how that would affect it.

    As far as I can tell the Linux builds are not affected.

    Cerberus X 2018-11-02 msbuild

    Note: I've not push any thing to get yet, until it can be figured out.
     
    Last edited: Nov 10, 2018
    MikeHart likes this.
  10. MikeHart

    MikeHart Administrator Staff Member Administrator Moderator

    Messages:
    1,231
    Likes Received:
    300
    Trophy Points:
    83
    CX examples and desktop/console targets work fine.
    My AGK target doesn't work as it can't find the MS tools, even if I set the path in the config file, like I do in the official version.

    EDIT: The copy_shared_test doesn't compile anymore. Can't find MSVC.
     
    Last edited: Nov 8, 2018
  11. MikeHart

    MikeHart Administrator Staff Member Administrator Moderator

    Messages:
    1,231
    Likes Received:
    300
    Trophy Points:
    83
    Will test the build script tommorow
     
  12. dawlane

    dawlane Active Member CX Code Contributor

    Messages:
    385
    Likes Received:
    161
    Trophy Points:
    43
    I didn't update that part, but replace the part calling MSBuild with
    Code (Cerberus X):
    1. Execute "~q"+tcc.compiler.Path+"/"+tcc.compiler.Executable+"~q /p:Configuration="+casedConfig+" "+buildpath+"\Template.sln"
    should fix that bit.

    Using the old method of setting the full path to msbuild will not work. The compiler detection should construct the full path.
    I'm also thinking that may be json file should be used for parts instead of trying to rip information from the command line.

    The sharedtrans application will always report that it cannot find Visual Studio when compiling with MinGw. And that's because it's not reading the config text files to get the additional bit of data that transcc uses. I may have to add some addition parameter passing.

    The odd bug where exceptions are not being caught is going to be a bit of a pain to try to locate and fix.
     
    MikeHart likes this.
  13. MikeHart

    MikeHart Administrator Staff Member Administrator Moderator

    Messages:
    1,231
    Likes Received:
    300
    Trophy Points:
    83
    Thanks, will look into this again.
     
  14. MikeHart

    MikeHart Administrator Staff Member Administrator Moderator

    Messages:
    1,231
    Likes Received:
    300
    Trophy Points:
    83
    Ok, tried the powershell rebuildall.ps1 command with MSVC2017

    .\rebuildall.ps1 -msbuild -vsversion "2017"

    or

    .\rebuildall.ps1 -msbuild

    gives an error when building trans.

    Code ( (Unknown Language)):
    1. ========= CERBERUS BUILD SCRIPT =========
    2. ========= NO WARRANTIES. USE AT OWN RISK =========
    3. ========= STARTED: 11/28/2018 20:05:31 =========
    4.  
    5. ========= Current parameters =========
    6. This script is at C:\users\mike\desktop\cerberus-msbuild\src.
    7. The Cerberus root directory is at C:\users\mike\desktop\cerberus-msbuild.
    8. MinGw is set to C:\Mingw.
    9. Qt SDK is set to \5.9.2\msvc2017.
    10. Visual Studio is at C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC.
    11. w64 Static build switch is set to False.
    12. MSBuild switch is set to True.
    13. 32/64 build swith is set to False.
    14. This script is running on a 64 bit operating system. Therefore 64 bit binaries will be built.
    15. QtSDK tools changed from \5.9.2\msvc2017 to \5.9.2\msvc2017_64
    16.  
    17. ========= Testing build environment =========
    18. Added C:\Mingw, C:\Qt\5.9.2\msvc2017_64 and C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\15.0\Bin; to current shell environment path.
    19. Testing for qmake
    20. EXECUTING: qmake -v
    21. OK: Executed
    22. QMake version 3.1 Using Qt version 5.9.2 in C:/Qt/5.9.2/msvc2017_64/lib
    23.  
    24. Testing for Visual Studio VC
    25. VC Tool chain enviroment already initialised.
    26.  
    27. ========= Checking for config.winnt,txt =========
    28. OK: Found config.winnt.txt
    29.  
    30. ========= Extracting archives if any =========
    31. Extracting C:\users\mike\desktop\cerberus-msbuild\src\archives\libs\winnt\shared_libs_win.zip to C:\users\mike\desktop\cerberus-msbuild\libs\shared
    32. Extracting C:\users\mike\desktop\cerberus-msbuild\src\archives\libs\winnt\static_libs_mingw.zip to C:\users\mike\desktop\cerberus-msbuild\libs\static
    33. Extracting C:\users\mike\desktop\cerberus-msbuild\src\archives\libs\winnt\static_libs_visualstudio.zip to C:\users\mike\desktop\cerberus-msbuild\libs\static
    34. Extracting C:\users\mike\desktop\cerberus-msbuild\src\archives\licences.zip to C:\users\mike\desktop\cerberus-msbuild\libs\
    35. Extracting C:\users\mike\desktop\cerberus-msbuild\src\archives\icons.zip to C:\users\mike\desktop\cerberus-msbuild\src\archives\
    36.  
    37. ========= Building transcc from Cerberus Source =========
    38. Please wait
    39. A backup of transcc_winnt has been detected. Do you wish to restore to rebuild transcc? (Y)es/(N)o
    40. Answering No will replace the backup with the current version of transcc_winnt.
    41. Answer Y/N: y
    42. Deleting current.......
    43. OK: Deleted C:\users\mike\desktop\cerberus-msbuild\bin\\transcc_winnt.exe
    44. Restoring back up copy....
    45. OK: Copied C:\users\mike\desktop\cerberus-msbuild\bin\transcc_winnt.exe.bak to C:\users\mike\desktop\cerberus-msbuild\bin\transcc_winnt.exe
    46. EXECUTING: C:\users\mike\desktop\cerberus-msbuild\bin\transcc_winnt.exe -msize=64  -msbuild -vsversion="2017" -target="C++_Tool" -builddir="transcc.build_new" -clean -config="Release" "transcc\transcc.cxs"
    47. TRANS cerberus compiler V2018-11-02 preview
    48. Compiler detection
    49. WARNING: MinGW not found.
    50. Detected: 0 MinGW compilers.
    51. Check that you have MinGW installed And set up correctly If you are using it.
    52.  
    53.  
    54. Parsing...
    55. Semanting...
    56. Translating...
    57. Building...
    58.  
    59.  
    60. Deleted old icon resource file C:/users/mike/desktop/cerberus-msbuild/src/transcc/transcc.build_new/cpptool/msvc2017/Release64/main.res
    61. Copying icon from Cerberus resource directory to C:\users\mike\desktop\cerberus-msbuild\src\transcc\transcc.build_new\cpptool/icon.ico
    62.  
    63.  
    64. USING COMPILER TOOL CHAIN: Visual Studio 2017
    65. Over-ride of CC_USE_MINGW via the command line option -msbuild.
    66. Over-ride of Visual Studio default 2017 via command line option -vsversion.
    67.  
    68. Location: C:\Program Files (x86)/Microsoft Visual Studio/2017/Community/MSBuild/15.0/Bin/
    69. Over-ride of CC_VS_MSIZE_WINNT via command line option. MSIZE is now set to 64bit.  An error will be generated by the compiler if it's not supported.
    70. Executing C:\Program Files (x86)/Microsoft Visual Studio/2017/Community/MSBuild/15.0/Bin//msbuild.exe /p:Configuration=Release64 /p:projectname="main_winnt" /p:CerberusPath="C:/users/mike/desktop/cerberus-msbuild" /p:CerberusLibraryPaths="" /p:CerberusIncludePaths="" /p:CerberusLinker="" /p:CerberusIncludes="" /p:CerberusDefines="" /p:CerberusCompilerOpts="" /p:CerberusResDefs="__USE_ICON__"
    71. Microsoft (R)-Buildmodul, Version 15.7.179.6572 für .NET Framework
    72. Copyright (C) Microsoft Corporation. Alle Rechte vorbehalten.
    73.  
    74. Die Projekte in dieser Projektmappe werden nacheinander erstellt. Um eine parallele Erstellung zu ermöglichen, müssen Sie den Schalter "/m" hinzufügen.
    75. Der Buildvorgang wurde am 28.11.2018 20:05:46 gestartet.
    76. Projekt "C:\users\mike\desktop\cerberus-msbuild\src\transcc\transcc.build_new\cpptool\msvc2017\main.sln" auf Knoten "1"
    77. (Standardziele).
    78. ValidateSolutionConfiguration:
    79.   Die Projektmappenkonfiguration "Release64|x64" wird erstellt.
    80. Das Projekt "C:\users\mike\desktop\cerberus-msbuild\src\transcc\transcc.build_new\cpptool\msvc2017\main.sln" (1) erstel
    81. lt "C:\users\mike\desktop\cerberus-msbuild\src\transcc\transcc.build_new\cpptool\msvc2017\main.vcxproj" (2) auf Knoten
    82. "1" (Standardziele).
    83. PrepareForBuild:
    84.   Das Verzeichnis "Release64\" wird erstellt.
    85.   Das Verzeichnis "Release64\main_winnt.tlog\" wird erstellt.
    86. InitializeBuildStatus:
    87.   "Release64\main_winnt.tlog\unsuccessfulbuild" wird erstellt, da "AlwaysCreate" angegeben wurde.
    88. ClCompile:
    89.   C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\bin\HostX86\x64\CL.exe /c /Zi
    90.    /nologo /W0 /WX- /diagnostics:classic /O2 /Oi /D _CRT_SECURE_NO_WARNINGS /D __VISUAL_STUDIO__ /D _WIN32 /D WIN32 /D
    91.   NDEBUG /D _CONSOLE /D _LIB /D _UNICODE /D UNICODE /Gm- /EHsc /MD /GS /Gy /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inl
    92.   ine /Fo"Release64\\" /Fd"Release64\vc141.pdb" /Gd /TP /FC /errorReport:queue /bigobj ..\main.cpp ..\stdafx.cpp
    93.   main.cpp
    94.   stdafx.cpp
    95.   Code wird generiert...
    96. ResourceCompile:
    97.   rc.exe /D __USE_ICON__ /D _UNICODE /D UNICODE /D _UNICODE /D UNICODE /l"0x0409" /nologo /fo"Release64\main.res" ..\ma
    98.   in.rc
    99.   TRACKER : Fehler TRK0005: Fehler beim Suchen von "rc.exe". Das System kann die angegebene Datei nicht finden.
    100.  
    101.  
    102. C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\VC\VCTargets\Microsoft.CppCommon.targets(1454
    103. ,5): error MSB6006: "rc.exe" wurde mit dem Code 5 beendet. [C:\users\mike\desktop\cerberus-msbuild\src\transcc\transcc.
    104. build_new\cpptool\msvc2017\main.vcxproj]
    105. Die Erstellung des Projekts "C:\users\mike\desktop\cerberus-msbuild\src\transcc\transcc.build_new\cpptool\msvc2017\main
    106. .vcxproj" ist abgeschlossen (Standardziele) -- FEHLER.
    107.  
    108. Die Erstellung des Projekts "C:\users\mike\desktop\cerberus-msbuild\src\transcc\transcc.build_new\cpptool\msvc2017\main
    109. .sln" ist abgeschlossen (Standardziele) -- FEHLER.
    110.  
    111.  
    112. Fehler beim Buildvorgang.
     
  15. dawlane

    dawlane Active Member CX Code Contributor

    Messages:
    385
    Likes Received:
    161
    Trophy Points:
    43
  16. MikeHart

    MikeHart Administrator Staff Member Administrator Moderator

    Messages:
    1,231
    Likes Received:
    300
    Trophy Points:
    83