GSOC 19 Final report  |  Forum thread

Work summary for the whole project.

Warning: this page is a directly linkable part of a simplistic blog page with more content.

GSOC 2019 Final Report - The Terasology Foundation's The render DAG enhancements

Let me start by explaining what this project has been about.
The rendering pipeline of Terasology's had been undergoing major re-factoring for a past few years before this GSOC. When I got to it, it was already based on a graph of rendering nodes and consisting of a solid code-base for queuing graphics API calls, neatly abstracted data representation and centralized data management systems for working with them. Despite all this, the render graph still lacked node to node data flow concept as it only used aforementioned centralized system, hard-coded data-links and so on. Dependencies were therefore hard-coded with no way to work with them dynamically. This was one of the main blockers for moving all rendering nodes to a module so the render graph can be entirely built only from module-based nodes. This was one of the main goals - modular rendering pipeline. Hence this project was based around architectural re-factoring of the rendering.

Planned deliverables vs final product in the course of action:
After the project acceptance and some general planning the main goal had been set on "moving all rendering nodes to separate modules - BasicRendering and AdvancedRendering". In order to be able to do this, I first had to draft a way to represent data dependencies between nodes. All this (and all architectural changes - i.e. except modules) had been drafted under pull-request So I came up with a concept of dependencyConnections. Laying foundations for these connections, reworking main classes for RenderGraph, WorldRenderer and all nodes used in this process, took me first month. Second month was dedicated to moving all nodes into respective modules and most importantly rewriting whole pipeline to run on the new architecture without any drawbacks/changes in the graphical fidelity.
Another goal we wanted to achieve was to introduce reusable nodes which would enable compositing. Though this seems like something rather useful in the future when there is a UI to mess with the rendering pipeline in the game, it turned out, during the second half of the project, that there were more pressing issues and sadly did not fit the scope. I tried to provide some generic methods for coder-friendly inserting of nodes into a specific place in the pipeline but I quickly bumped into layer of problems which will yet have to be solved. Though if you take a look at my dagTestingModule branch linked below, there is a tinting submodule that you can use to tint buffers. This is one of the areas that will require more focus in the future.
Speaking of more pressing issues, one of the things I deemed really important during the last month was configuring the rendering modules in game. Providing a working proof of concept for managing rendering modules took me most of the third month's time and is a part of the aforementioned PR.

Final Product:

  • I provided architectural foundation for expressing inter-node data, or other, dependencies and enabled data flow trough the rendering graph. (took place during first month)
  • I transformed the rendering architecture to this new concept and extracted whole rendering graph building and all nodes into separate modules while maintaining same graphical capabilities and output. This sort of enabled upgraded version of an in-game command for debugging - not only can we now reconnect any frame-buffer to output to take a look at different pipeline stages, but we can now reconnect any buffer/buffer-pair to any other node during run-time. (took place during second month)
  • I provided a working concept of rendering module subsystem taking care of rendering modules and coupled that with a game UI. This provides user with the ability to do the following during (so far) game creation process (other than in code): Select rendering classes from each activated module that will be used and order their initialization. (took place during third month)

Where the work must continue (follow up issues will be provided accordingly):
Basic and Advanced Rendering modules use hacky tactics to enable absence of the Advanced rendering module in each BasicRendering-based node which uses AdvancedRendering-based node's output. When someone takes on the quest of splitting feature sets (nodes with more than one graphical-effect responsibility) into more atomic units, he will have to deal with this fact. This part will also have to leverage another project's concept of FlexibleConfigs to deal with lesser graphical capabilities when AdvancedRendering module it not activated in the sense of not enabling user to try and activate them.
Another area that will require focus is the rendering configuration. Together with another project dealing with FlexibleConfigs, this part of rendering needs to save configurations on a per-game basis and enable using either default values or the saved ones.
Further on a UI for the rendering pipeline to dynamically configure it while seeing what's happening is feasible. This was at first in my proposal, but we didn't go for this in this project.
Regarding RenderGraph, AbstractNode and other fundamental parts of the rendering - there's still leftovers to go trough but it's time consuming since verifying correct behavior at this point is still hard. Namely cleaning the APIs.

Personal comment:
I'm personally quite happy with the amount of working code I've ended up with. Since getting around the huge code-base was a little intimidating and every new concept required working with A) previously unseen part of the code and B) a lot of issues my mentor and I hadn't foreseen. I'm a little sad I couldn't manage to create some examples of how to use this by module creators together with a guide, but this is something that will come sooner or later when it's time.

I would also like to thank Vampcat, Cervator, eviltak and others for volunteering their free time to help me and this project.

Last but not least, all changes in the form of weekly reports are on this blog/forum-thread, which has been dedicated to them.

All my changes and repos in order of priority:

All links for this project:

Relevant images of a Sketchboard I used to get around and a provided UI for rendering modules configuration

Rendering Module Configuration settings

Graph of the render dag I used to mark my changes. It links to the online version of the said graph where you can take a look at the legend. Bear in mind it might not be entirely accurate to the state of code. The previous version is here. sketchboard-graph

Last Edit by Dave2S on August 26, 2019

Comments   0

Comments section