Implemented Looking at improving GLFW build system

dawlane

Active Member
CX Code Contributor
Joined
Jun 21, 2017
Likes
198
#1
As I've got some time today I thought I would begin to have a look at improving the build systems currently employed by Cerberus for GCC Windows (for now).

The idea is that all dynamic link libraries should be stored in a single lib folder with the usual directory structure (e.g. lib/Win32, lib/Win64). This is to try to cut out the number of duplicates that can happen in the targets directory. The transcc compiler then has two jobs:
  1. Scan the new lib/Win32/64 for linking. The use of .LIB files is not guaranteed to work between compilers, even between versions of Visual Studio, but dlls should.
  2. Reads a pre-processor directive for the libraries to copy over to the final build directory ready for deployment.
Before Mark abandoned MonkeyX, he did add some new GCC pre-processor directives, but never completed them.

These are: #GLFW_GCC_CC_OPTS and #GLFW_GCC_LD_OPTS, but he never update the Makefiles for Linux or included being able to pass library linking.
So I will fix and add:
#GLFW_GCC_CC_OPTS
#GLFW_GCC_LD_OPTS
#GLFW_GCC_LIB_OPTS
#GLFW_COPY_LIBS

The next task is to create a 'rebuildall' script in Power-shell which if I recall has been around since Vista. I've got this partially done, but cserver and the launcher need to be sorted out.
 

muruba

Active Member
CX Code Contributor
3rd Party Module Dev
Patreon Silver
Joined
Jul 5, 2017
Likes
57
#2
Sounds good, also I am wondering if Win32 should be removed, just to make contributors life easier, with all love to 32 bit (and nostalgia) it seems really obsolete at this stage. The recent CX GLFW build doesn't run on 32 anyways as included CURL dlls are 64-bit only...
 

dawlane

Active Member
CX Code Contributor
Joined
Jun 21, 2017
Likes
198
#4
I wouldn't say that it will speed up build times. Just make it less cumbersome to use and frees the end user from having to use MinGW versions less than 5 or install OpenAL to the system. I've always thought that MonkeyX's build system was poorly conceived and Monkey2's doesn't look any better.
 

dawlane

Active Member
CX Code Contributor
Joined
Jun 21, 2017
Likes
198
#5
Not sure if cserver should be rewritten in Qt or if Cerberus it's self is up to the job for all three desktop platforms. The less reliance on BlitzMax the better. The launcher would have to be in C/C++ for dealing with the Windows registry.
 

Martin

Administrator
Staff member
Joined
Jun 19, 2017
Likes
147
Location
Germany
#6
Thanks, looks good! You're right with cserver. I had a quick try but it did not work out of the box to build it with CX.
 

MikeHart

Administrator
Staff member
Joined
Jun 19, 2017
Likes
427
Location
Germany
#7
The launcher is cx so i don't think you need to rewrite that. I would rather like to see enhancements to cx so cserver could be build with cx.
 

dawlane

Active Member
CX Code Contributor
Joined
Jun 21, 2017
Likes
198
#8
The launcher is cx so i don't think you need to rewrite that. I would rather like to see enhancements to cx so cserver could be build with cx.
Only for Linux and OS X. It doesn't set the registry keys for the application in Windows.
As for cserever; that would have to be looked at, but it would be a question of why Mark himself never bothered with implementing it.
 

dawlane

Active Member
CX Code Contributor
Joined
Jun 21, 2017
Likes
198
#9
OK. Here's what I've got for the launcher for Windows. Any thoughts?
C++:
#include <windows.h>
#include <iostream>
#include <string>
#include <sstream>

using namespace std;

const wstring nullStr(L"");
const long int CXL_VERSION = 68;
const string build("2017-9-21");

// Convert to wstring type
template <typename T>
wstring to_wstring(T const & value)
{
   wstringstream ss;
   ss << value;
   return ss.str();
}

// Set a registry key value
bool SetKey(HKEY root, const wstring &subkey, const wstring &name, const wstring &value)
{
    HKEY hKey;
    if(RegCreateKeyExW(root, subkey.c_str(), 0, 0, REG_OPTION_NON_VOLATILE, KEY_ALL_ACCESS, 0, &hKey, NULL) == 0)
    {
        const wchar_t *p = value.c_str();
        size_t sz = wcslen(p)*2+2;

        if(RegSetValueExW(hKey, name.c_str(), 0, REG_SZ, (LPBYTE)(p), sz) == 0)
        {
            RegCloseKey(hKey);
            return true;
        }
        RegCloseKey(hKey);
    }
    return false;
}

// Get a registry key value
wstring GetKey(HKEY root, const wstring &subkey, const wstring &name)
{
    HKEY hKey;
    if(RegOpenKeyExW(root, subkey.c_str(), REG_OPTION_NON_VOLATILE, KEY_QUERY_VALUE, &hKey) == 0)
    {
        wchar_t buffer[MAX_PATH*2];
        DWORD bufferSize = 0;
        DWORD dataType = 0;
        memset(buffer, 0, sizeof(buffer));
        bufferSize = sizeof(buffer);

        if(RegQueryValueExW(hKey, name.c_str(), 0, &dataType, (LPBYTE)(buffer), &bufferSize) == 0)
        {
            if(dataType == REG_SZ)
            {
                RegCloseKey(hKey);
                buffer[MAX_PATH - 1] = 0;
                return buffer;
            }
        }
        RegCloseKey(hKey);
    }
    return nullStr;
}

// Main program entry point
int main(int argc, char* argv[])
{
    printf( "Cerberus Launcher %s\n", build.c_str() );

    // Get current launcher location
    wstring app = to_wstring(argv[0]);

    // Put all data into the Current user registry area
    HKEY root = HKEY_CURRENT_USER;

    // File types supported and storage location
    wstring type_cerberusPath (L"Software\\Classes\\.cxs");
    wstring type_monkeyPath (L"Software\\Classes\\.monkey");
    wstring tool_path (L"Software\\Classes\\cerberus-x.com");

    /* Check that the version hasn't changed. Instead of checking for a greater version number,
       check to see if the version numbers differ. Doing it this way makes it so that the end
       user can downgrade as well as upgrade. Also check to see if the Cerberus directory has been moved
    */
    if((CXL_VERSION != _wtoi(GetKey(root, tool_path+wstring (L"\\Version"), nullStr).c_str())) || (app != GetKey(root, tool_path+wstring (L"\\Location"), nullStr).c_str()))
    {
        wstring shell(L"\"");
        shell.append(app);
        shell.append(L"\" \"%1\"");

        SetKey(root, tool_path, nullStr, wstring(L"Simple Cerberus IDE"));
        SetKey(root, tool_path+wstring (L"\\Version"), nullStr, to_wstring(CXL_VERSION));
        SetKey(root, tool_path+wstring (L"\\DefaultIcon"), nullStr, app);
        SetKey(root, tool_path+wstring (L"\\Shell\\Open\\Command"), nullStr, shell);
        SetKey(root, tool_path+wstring (L"\\Location"), nullStr, to_wstring(app));  // Used if the Cerberus directory is relocated

         // Saves having to rebuild this application launcher if a new IDE is added. The new IDE checks this and updates it for its own use.
        SetKey(root, tool_path+wstring (L"\\CurrentIDE"), nullStr, wstring(L"bin\\Ted.exe"));
    }

    // Set application file associations
    if((GetKey(root, type_cerberusPath, nullStr) == nullStr)||(GetKey(root, type_monkeyPath, nullStr) == nullStr))
    {
        SetKey(root, type_cerberusPath, nullStr, wstring (L"cerberus-x.com"));
        SetKey(root, type_monkeyPath, nullStr, wstring (L"cerberus-x.com"));
    }

    // Construct the command line
    wstring args(GetKey(root, tool_path+wstring (L"\\CurrentIDE"), nullStr));
    for(int i = 1; i < argc; i++)
    {
        args.append(L" \"");
        args.append(to_wstring(argv[i]));
        args.append(L"\"");
    }

    // Use create process to start the IDE with command line arguments.
    STARTUPINFOW         siStartupInfo;
    PROCESS_INFORMATION piProcessInfo;

    memset(&siStartupInfo, 0, sizeof(siStartupInfo));
    memset(&piProcessInfo, 0, sizeof(piProcessInfo));

    siStartupInfo.cb = sizeof(siStartupInfo);

    if(CreateProcessW(NULL,
                    (LPWSTR)args.c_str(),
                    NULL,
                    NULL,
                    FALSE,
                    CREATE_DEFAULT_ERROR_MODE,
                    NULL,
                    NULL,
                    &siStartupInfo,
                    &piProcessInfo) == FALSE)
                    {
                        // Issue some sort of error message
                        printf("Cerberus Launcher %s\n", build.c_str());
                        wprintf(L"Failed on command:\n%ls\n", args.c_str());
                        printf("Press Return to exit\n");
                        std::cin.ignore();
                    }

    CloseHandle(piProcessInfo.hThread);
    CloseHandle(piProcessInfo.hProcess);
    return 0;
}
 
Last edited by a moderator:

dawlane

Active Member
CX Code Contributor
Joined
Jun 21, 2017
Likes
198
#10
And here is a Windows Powershell script. NOTE cserver still to sort out.
Code:
# Windows Powershell build script
# TO DO: The cserver needs to be rewritten so it doesn't need BlitzMax, currently the Cerberus version doesn't work.
param(
    [string]$mingw = "C:\Mingw",
    [string]$qtsdk = "C:\Qt\5.5\msvc2013_64",
    [string]$visualstudio = "C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC"
)

# NOTE: This script requires that the execution policy for the current user be set to unrestricted.
# Open power shell as administrator and use:
#    get-executionpolicy -list
#    set-executionpolicy -scope currentuser unrestricted
# If the file is still blocked use:
#    unblock-file -path "full_path_to_this_script"
# You should reset the execution policy back to it's original state e.g.:
#    set-executionpolicy -scope currentuser undefined

clear

function ErrorMsg($msg){
    Write-Host $msg -BackgroundColor Red -ForegroundColor Yellow
    exit 1
}

# Set compiler and sdk paths
if(-not (Test-Path "$mingw")){
    ErrorMsg "ERROR: No valid Windows GCC compiler tools installed!"
} else {
    $mingwPath = $mingw+";" + $mingw + "\bin;" + $mingw + "\include;" + $mingw + "\lib;"
}

if(-not (Test-Path "$qtsdk")){
    ErrorMsg "ERROR: No valid Qt SDK installed!"
} else {
    $qtsdkPath = $qtsdk + ";" + $qtsdk + "\bin;" + $qtsdk + "\include;" + $qtsdk + "\lib;" + $qtsdk + "\plugins;"
}

$env:Path = $qtsdkPath + $mingwPath + [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User")

# Add Visual Studio stuff. Note that this is cumulative and you will end up with an error that cmd's input line is too long, so just open and start another power shell session.
if(-not (Test-Path "$visualstudio")){
    ErrorMsg "ERROR: No valid Visual Studio tools installed!"
} else {
    pushd "$visualstudio"
    #if(Test-Path "$visualstudio\vcvarsall.bat"){
        cmd /c "vcvarsall.bat x86_amd64&set" |
        foreach {
            if ($_ -match "=") {
                $v = $_.split("="); set-item -force -path "ENV:\$($v[0])"  -value "$($v[1])"
            }
       # }
    }
    popd
}
#

# Build transcc first
if(Test-Path $mingw\bin\g++.exe)
{
    write-host "Building transcc"
    write-host "Please wait"
    if(Test-Path ..\bin\transcc_winnt.exe){
        remove-item ..\bin\transcc_winnt.exe -force
    }
    g++ -Wall -fexceptions -O3 -DNDEBUG -o ..\bin\transcc_winnt transcc\transcc.build\cpptool\main.cpp -lpthread -s

    write-host "Building launcher"
    if(Test-Path ..\Cerberus.exe){
        remove-item ..\Cerberus.exe -force
    }

    windres launcher\resource.rc -O coff -o launcher\res.o
    g++ -Wall -fexceptions -Os -DNDEBUG -o ..\Cerberus.exe launcher\codeblocks\launcher.cpp launcher\res.o -ladvapi32 -s
    if(-Not (Test-Path ..\Cerberus.exe)){
        ErrorMgs "ERROR: Failed to build Cerberus launcher."
    }
} else {
    ErrorMsg "ERROR: No valid Windows GCC C++ compiler installed!"
    }

# Only continue if transcc was successfully built
if(Test-Path ..\bin\transcc_winnt.exe){
 
    # Before continuing, fix the location of MinGw in \bin\config.winnt.txt so that transcc knows where it is.
    $file = '..\bin\config.winnt.txt'
    $regex = '(?<=MINGW_PATH=")[^"]*'
    (Get-Content $file) -replace $regex, $mingw | Set-Content $file

    #Make makedocs
    write-host "makedocs"
    if(Test-Path ..\bin\makedocs_winnt.exe) {
        remove-item ..\bin\makedocs_winnt.exe -force
    }

    ../bin/transcc_winnt -target="C++_Tool" -builddir="makedocs.build" -clean -config="release" +CPP_GC_MODE=0 "makedocs\makedocs.cxs"
    move-item makedocs\makedocs.build\cpptool\main_winnt.exe ..\bin\makedocs_winnt.exe
    remove-item makedocs\makedocs.build -force -recurse

    if(-Not (Test-Path ..\bin\makedocs_winnt.exe)){
        ErrorMsg "ERROR: Failed to build makedocs!"
    }

    <#Make cserver
    if(Test-Path ..\bin\cserver_winnt.exe) {
        remove-item ..\bin\cserver_winnt.exe -force
    }
    write-host "building cserver"
    ../bin/transcc_winnt -target="Desktop_Game_(Glfw3)" -builddir="cserver.build" -clean -config="release" +CPP_GC_MODE=1 "cserver\cserver.cxs"
    Move-Item cserver\cserver.build\glfw3\gcc_winnt\Release64\CerberusGame.exe ..\bin\cserver_winnt.exe
    #Remove-Item cserver\cserver.build -force -recurse
    #>
 
} else {
    ErrorMsg "Error: Failed to start transcc. Was it built successfully?"
}

#Make ted
write-host "building ted"
if(Test-Path ..\bin\Tedt.exe){
    Remove-Item ..\bin\Ted.exe -Force
    Remove-Item ..\bin\*.dll -Force
    Remove-Item ..\bin\plugins -Force -Recurse
}

if(Test-Path build-ted-Desktop-Release) {
    Remove-Item build-ted-Desktop-Release -force -recurse
}

New-Item build-ted-Desktop-Release -type directory
cd build-ted-Desktop-Release
qmake -spec win32-msvc2013 ../ted/ted.pro
nmake -f Makefile.Release
cd ..
Remove-Item build-ted-Desktop-Release -force -recurse

# Deploy and clean
windeployqt --no-svg --no-angle --no-compiler-runtime --no-system-d3d-compiler --no-quick-import --no-translations --core ..\bin\Ted.exe
$folders = "audio","bearer","imageformats","mediaservice","playlistformats","position","printsupport","sensors","sensorgestures","sqldrivers","opengl32sw.dll"
foreach($folder in $folders){
    Remove-Item ..\bin\$folder -Force -Recurse
}

exit 0
 
Last edited by a moderator:

Martin

Administrator
Staff member
Joined
Jun 19, 2017
Likes
147
Location
Germany
#11
Wow that's awesome! Will look into this, thanks! Probably next week as I'm going to have a full weekend this time...
 

dawlane

Active Member
CX Code Contributor
Joined
Jun 21, 2017
Likes
198
#13
Well cserver would work if creating a new process didn't leave a zombie when closed!
Update.
Building Html5 calling transcc via a terminal works as it should, but running from Ted keeps one thread alive, so I will have to investigate tomorrow if I get a chance. Could be a bug.

Changes:
modified the checks for Linux/MacOS and changed Execute "~q"+AppPath+"~q GO >/dev/Null 2>/dev/Null &"
to
Code:
#If HOST="winnt"
    Execute "start ~q~q ~q"+AppPath+"~q GO  >NUL 2>NUL &"
#Else
    Execute "~q"+AppPath+"~q GO  >/dev/Null 2>/dev/Null &"
#End
 
Last edited:

dawlane

Active Member
CX Code Contributor
Joined
Jun 21, 2017
Likes
198
#14
I've just push what I've done so far on to my fork of Cerberus. There is still the issue with Ted and that zombie to solve.
I wouldn't say that I'm rusty with the whole git thing, but it took me some time to figure out how to solve a merge conflict.
 

dawlane

Active Member
CX Code Contributor
Joined
Jun 21, 2017
Likes
198
#16
Martin will look into how to add it to our repository
If I remember, I have to send a git pull request. But I will do that when I've sort out Ted. For the time being I would just clone my repo, switch to the develop branch and see if the new power-shell script works. Then it would be a case of using the diff and patch tools.

Edit: I've also notice a weird thing with cserver not displaying text.
 

dawlane

Active Member
CX Code Contributor
Joined
Jun 21, 2017
Likes
198
#17
Looks like the issue with Ted as far as I can tell is related reading the standard I/O streams. They time out, thus leaving a zombie thread alive. The way I have found to get round it that appears to work, is to make the proc variable in cserver.cxs local to the function, create the process and add a while loop that checks to see if the process is still running.

I've update all files to my fork.
 
Last edited:

dawlane

Active Member
CX Code Contributor
Joined
Jun 21, 2017
Likes
198
#19
Not sure where I should concentrate on next.
  1. Fix it so Visual Studio will do the same thing as the MinGW GCC version
  2. Look at improving Angle for GCC and Visual Studio
  3. Implement the use of being able to use custom shared libraries on Linux.
Edit: What's done so far

#GLFW_GCC_CC_OPTS < For both Windows and Linux
#GLFW_GCC_LD_OPTS < For both Windows and Linux
#GLFW_GCC_LIB_OPTS < For both Windows and Linux
#GLFW_COPY_LIBS < For MinGW/TDM for now

New Windows Launcher written in C/C++.
Cerberus CServer code should now work with windows.
Both of the above means that the BlitzMax source is no longer needed for Windows builds.

Added a Power-shell rebuild all scrip that does the same thing and a bit more than the one use for Linux/OS X.

As I've implement #GLFW_GCC_LIB_OPTS and #GLFW_COPY_LIBS, it solves a few problems and adds flexibility, so libcurl no longer needs to be hard coded directly into any make files. You just add the necessary options to the source code that uses it. This means that if your application doesn't use curl, you don't have to distribute unnecessary shared libraries, or in the case of Linux, the end user having to have curl installed (any way, it's there as part of the default install on practically every distribution).
 
Last edited:

dawlane

Active Member
CX Code Contributor
Joined
Jun 21, 2017
Likes
198
#20
anything that speeds up desktop building will be welcome!
I may have an idea on how to speed up builds for GLFW. When the launcher is first run, it builds the standard set of support libraries and places them in the new central lib directory. It solve two things:
  1. Reduces the number of times the sources are duplicated in the target templates and each project.
  2. As these are pre-built, there would be no need to keep rebuilding them each time.
It would have been nice if Mark had though of this at the start.

Edit: I think I will add that to the top of the list before doing the other bits.
 
Top Bottom