FS22 - TestRunner Tool for Testing Mods

[Heizer]

Tool for testing Farming Simulator 22 Mods
As with FS19 we are happy to provide our Mod Testing Tool updated for FS22.

TestRunner_public.exe

• checks a mod and shows various errors and violations of ModHub guidelines
• provides test result as html and xml document
• tests are structured in multiple modules/groups
• tool is also used by our QA internally (with an extended feature-set)

Goals

• speed up the process of testing mods by helping the modder to identify and fix problems (before submitting to ModHub)
• detect various errors and provide hints how to fix them
• ensure ModHub guidelines are met
• improve performance and minimize overall mod file-size and memory usage

Key Features / What it does (selected examples)

• check if the descVersion matches current patch level
• check for xml parsing errors
• warn about wrongly exported dds textures
• warn about missing or obsolete files
• show duplicate files (within the mod and in the base game)
• provide information about shader problems, which need fixing for a console release
• and much more, for detailed explanations see "Test Cases" section
• all found problems will be provided in form of an HTML document and and xml file
  Example Result HTML

Requirements for using the tool

• Farming Simulator 22 (min. version 1.8)
• GIANTS Editor (min. Version 9.0.3)
  => the path to game installation needs to be correctly set within the editor
• latest version of the Farming Simulator 22 TestRunner (0.8.8)

Disclaimer

• the tool is still in beta status, some tests might show errors which are none (so called "false positives")
  particularly "unused files" reported by module "ObsoleteFiles"
• please report any cases where you think the tool outputs errors which are false positives
  when making a bugreport please include testrunner log, the result files (xml and html) and if possible the mod itself (or its id on the ModHub), so we can reproduce the issue
  ideally use the "TestRunner" project on our public Bugtracker for reporting any bugs

Instructions / How to use

1. drag and drop the directory of an unzipped mod onto TestRunner_public.exe
   • note: folder of the mod does not have to be in the same directory as the TestRunner
   • a command-line window will open displaying current progress
   • if something went wrong an error message is displayed giving details on the problem
     all outputs are also saved in the "TestRunner.log"
2. if requirements are met the tool will start to analyze the mod
   • the command line will display some information on the current progress
     unless there is a "critical" message, this can be ignored
   • GIANTS Editor will be automatically opened and closed during the process
   • note: depending on the size of the mod and the speed of the hard drive the process may take several minutes
3. when the test process has finished, the message "Finished execution" will be displayed inside the command line
   • a xml and html file will be generated in the same directory as the TestRunner
   • files are named after the directory name of the tested mod and contain the result of the test
     • html file only contains all the found errors (per module)
     • the xml contains all errors and more detailed (meta)information
4. the generated html file will be automatically opened in your browser where you can check if errors were found in your mod
   • errors are outlined in red
   • explanations are in italics
   • instruction on how to correct errors are colored green

Test Cases and explanations

In the following the individual modules and their test cases are explained in detail

	Work in Progress xml-file well-formedness (only required by the game to a certain extent, but should be fulfilled nonetheless) Module DXTCheck: ensure dds files use correct DXT formats, suggest conversion to DXT1 where possible Module MipMapCheck: ensure textures for 3D objects have MipMaps, warn if texture has too low resolution Module TextureCheck: ensure textures have a resolution of 2^n Module I3DCheck: check reference integrity, obsolete lights/cameras, textureArray usage Module ModDescCheck: check a set of ModHub guidelines (descVersion, version number format, changelog, localization, ...) Module VehicleCheck: ensure specific vehicle xml elements are present (e.g. "washable") or absent (e.g. "dailyUpkeep" in vehicles) Module SlotCheck: calculate estimated number of used slots for each store item as a reference for the modder Module ObsoleteFiles: list obsolete source files not loaded by the game, list possibly unused textures, list duplicate files Module EditorCheck: load all i3d files contained in the mod and check for errors Module ShaderCheckEditor: warn if custom shaders are present in the mod nullifying console compatibility, raise errors if material configurations are used which are not available in the shader cache

Troubleshooting

• problem: game installation can not be found
  • reason: tools checks default install location and reads location from registry, make sure you didn't move the game after installation
  • solution: run in cmd (see instructions below) with argument -g <pathToGameInstallation> to manually set the path
    
• problem: editor installation can not be found
  • tools searches for installed Giants editors in giantsPackageRegistry xml located in appdata/local
  • solution: run in cmd (see instructions below) with argument -p $FS22_EDITOR$ <pathToEditorExe>
• problem: program closes down by itself/crashes
  • post/send TestRunner.log, if possible send the tested mod in a private message
  • if possible, run the tool from command line and add "--verbose" as an argument and send us the log
    this way more information will be included in the log-file and help us with debugging process

Instructions for using with command-line (cmd)

if you need to make additional settings (e.g. game installation cannot be found automatically), you can run the tool from a commandline

1. start a command-line or powershell window
   • Option A: click the address bar of the windows explorer window, type "cmd", hit enter
     this will open a command-line at the current directory of the explorer
   • Option B: hold shift and right click the background of the explorer window, select "Open Commandline window here" or "Open PowerShell window here"
     this will open a command-line/powershell at the current directory of the explorer
   • Option C: Press "Win" + "R", type "cmd" in the dialog, hit enter
     this will open a command line in your user profile
     => navigate to the location of the TestRunner_public.exe
2. type "TestRunner_public.exe" ("./TestRunner_public.exe" for PowerShell)
3. hit space and enter the path to your unzipped mod, e.g. "D:\mods\FS22_caseIHMagnum724Pro"
4. Optional: add additional arguments if required/desired
   
5. hit enter to start the test tool

If you find any bugs I suggest to report them on our public Bugtracker, as its more efficient to manage them there.
We are looking forward to your Feedback and hope the tool will support you creating mods and submitting them to ModHub.

[MrMarcel5]

Errors regarding "parseError: ('texdiag analyze regex failed'..." are likely to occur with textures converted by our texture converter.
If this is the case, these can be ignored for now.

[ALiEN JiM]

Errors regarding "parseError: ('texdiag analyze regex failed'..." are likely to occur with textures converted by out texture converter.
If this is the case, these can be ignored for now.

parseError also from store icons created by FS Icon Generator

[strauts6]

Is this a false positive error or is there a proper way to adding custom placeables to a map?

Code: Select all

<filenotfound referencedin="FS22_Dunalka/maps/mapUS/storeItems.xml">maps/placeables/waterFillTriggers.xml</filenotfound>

Test tool says that the xml files cant be found, but they are there and works perfectly in game.

[Minecraftchest1]

Command line tools. My favorite. Time to decide if I want to run tests out of CI/CD or not.

[martin)]

ERROR root - No game installation path set/found

[CavemanTechGaming]

Got this error today
ERROR root - 'NoneType' object has no attribute 'with_suffix'
AttributeError: 'NoneType' object has no attribute 'with_suffix'

What does this mean?

[xludo_modding]

hi i can't connect with the game to the testrunner ? Help

[liberoaro]

ERROR root - No game installation path set/found

ja mam ten sam problem

[Jarkozpl]

How to download it?

[TheSuBBie]

https://gdn.giants-software.com/

[doodoorolf]

Hallo zusammen
habe den neuesten Testrunner heruntergeladen und wollte einen MOD prüfen.
Der Testrunner startet und bleibt nach kurzer Zeit stehen und zeigt keine Reaktion.
Es öffnet sich der GE, jedoch der Mod wird nicht geladen und getestet.

LOG anbei des Testrunner.
https://image.giants-software.com/index ... codes=true

Welche Schritte muss ich unternehmen um den Testrunner zu verwenden.

Danke
LG

NACHTRAG:
Nach mehreren Versuchen zeigt er mir diese Fehlermeldung im GE.

Started 1 threads for threadpool 'TerrainPatchOccluderManager unique worker'
Error: Can't load resource 'C:/Users/G_tze Rolf/AppData/Local/Temp/_MEI162842/additionals/tmp_14836_loadI3DFiles.lua'.
Check for updates (https://gdn.giants-software.com)

Ein Update habe ich durchgeführt, leider ohne Erfolg
Die erwähnte Datei bei ERROR gibt es nicht in dem Verzeichnis

[CyTec502]

It's a stuff program. Does not work. Nothing can be done with it.

[LS-Lara]

It's a stuff program. Does not work. Nothing can be done with it.

It does exactly what it is supposed to do - if operated properly. What is the actual issue you encounter under which operational circumstances ?

[CyTec502]

It's a stuff program. Does not work. Nothing can be done with it.

It does exactly what it is supposed to do - if operated properly. What is the actual issue you encounter under which operational circumstances ?

It starts. Press a key. Then exit.