# Git

# What is Git?

Git (opens new window) is a widely adopted distributed version-control system. (DVCS (opens new window)) keeps track of changes applied to a collection of files. It is an actively maintained open-source project that is used in a wide variety of commercial and non-commercial software projects. It enables each contributing team member to work on different revisions of a project simultaneously, to manage changes, and to discuss them.
The following subsections will describe the advantages of using Git in ONE DATA Apps and will describe its basic principles and terminology. The subsequent sections will show how to use Git with ONE DATA Apps. The sections GitLab and Gitea will give a more detailed overview of both systems.

# Why Git?

  • Git tracks changes team members make
    Git tracks the changes made to files, creating a record of what has been done and by whom.
  • Git makes collaboration easier
    Git allows creating so-called branches, i.e., copies of the Project to
    work on without having to deal with concurrent changes by other people. After done work, changes can be brought backs to the original or main branch
    of the project, integrating these changes with those made by other Builders.
  • Git is great for quality-assurance
    After done work, it is possible to create a so-called merge request.
    The changes can be reviewed by another person before they are made available to
    others. This encourages a Four-eyes principle were only peer-reviewed and quality assured changes get to merged into the main branch.
  • Git helps to keep track of what is happening in a project
    With Git, the complete history of a project can be viewed. It thus is, for
    every line of code in the project, visible to who wrote or changed it and whom
    to ask if any question occurs. Also, it is possible to see in what context a certain
    part of code was originally written as well as the changes that have occurred in
    the past, allowing to get a better understanding.

The following subsections will describe the basic principles and terminology of Git at a high level. Some of this information is not necessary to work on Apps with Git, but it helps to understand the underlying principles.

# How to Git?

To be able to use Git, one of Git management tools (GitLab or Gitea) has to be set up by the DevOps on the desired instance. Then
special options for managing Apps will be visible to Apps' admins. This will be covered in detail later.

# Repositories

A Git repository (or repo for short) is a directory that contains all files of a project and the entire history of these files. When checking out a certain version of a project, Git will adjust the contents of the folder to the state that was present when the version was created.

In most cases, the repository is stored in some central location, for example, a
GitLab server. This is called the remote repository. It is possible to clone the repository from that location, creating a copy of it on a local machine
(called the local repository), allowing to make changes to the files of that local copy and push the changes back to the remote repository. This makes the changes available to other people, who can pull them into their own local repositories and continue working on this state of the project. Therefore, Git is called a distributed version control system as everyone is working on local copies of the repository and synchronizes those via the central copy by pushing and pulling.

Image from alias

# Commits

The state of a project can be seen as the history of all changes made to it so far. Git is keeping track of the sequence of changes made to the files instead of the content of the files themselves. Thus, it can use the history of the changes to reconstruct the state at any point in time. For Git to record the changes made to the project a so-called commit has to be made. A commit is a set of deletions and additions made to the files of a project since the last commit (the so-called parent commit). For every commit, a message can be specified, which should describe the changes it contains.
By making a commit a Snapshot of the current state of work is created which should appear in the project's history and that can be jumped back to later. It is not necessarily a stable state of the feature that is worked on but can be some work-in-progress state that should not be lost.

The entire series of commits made since the project was created is called the history. As every commit contains the difference to the previous commit, the current state of the project could be restored by applying one commit after the other.

# Branches

There are potentially many people working on different features in a project. To get other peoples commits and to wind up with work-in-progress changes it is required to synchronize with the remote repository. Ideally, one would want to work isolated from the changes made by others and only bring changes back to the main codebase once they are stable and quality-assured.
To allow such behavior, Git provides a functionality called branching. By branching away from the main history of a repository a copy of the original codebase in a new branch is created. Branches are clones of the main history of the project that can be worked on in parallel without being impacted by changes made to other branches or the main history of the project (which is itself a branch). Whenever work is completed, the history of a new branch can be integrated back into the main history.

The following figure depicts the concept of branching graphically. The lines represent branches and the circles represent the commits made on them. A branch (green) is created starting at a specific commit on the source branch (red). Then commits are made to this new branch. The history of that branch now consists of these new commits and the old commits of the source branch until the point where the branch was created. Newer commits made to the source branch after this point do not affect the history of the new branch. In the end, the new branch is brought back to the source branch, integrating its changes in its history. Now both the changes made to both branches are available in the source branch.

Image from alias

# Merging

Once a branch is created and work finished, the changes can be brought to the main branch. This is done by merging. While working on a local branch, changes may have been made to the main branch. These changes can potentially affect the same files one was working on. So when integrating the changes from a local branch, the changes from both branches must be joined. This is automatically done by Git if the changes do not affect the same spot in a file. As the files need to be changed by Git to bring the changes together, a so-called merge commit is created (see figure above).
In case the same part of the file was changed in both branches, the changes can not be automatically merged as Git does not know which change to prefer. In this case, a merge conflict will be reported and a human agent needs to decide which of the conflicting changes should be kept or how they can be unified. This can be done in any text editor or with one of the many merge tools available. This section will describe how conflicts can be resolved via the GitLab web interface.

# GitFlow

GitFlow is a branching model that is widely used in git repositories of many software projects. It encourages a clean separation of feature development and thus improving clarity and reducing the risk of conflicts. GitFlow tries to model the process of software development with branches. It defines conventions on which branches to use, how they are related, and what purpose these branches have. A GitFlow repository has the following branches:

  • master: This is the live-branch of a project containing the production code. Only quality assured code that is ready to go live (e.g., into production) should be pushed to it.
  • develop: This branch contains the current development state that is not live yet. Only finished and quality-assured work should be pushed to this branch. It should always be in a state that is ready to go live.
  • feature/* The feature branches in the repository contain the development changes for a certain feature. For each feature or bugfix that should be implemented, a feature branch is created starting from the develop branch. Then the changes for the work item are created on it. Once the work is finished and quality-assured, the feature-branch is merged back to develop. The name of feature-branch always starts with feature/ followed by the name or description of the feature that it includes.
  • release Before going live with the state of the current develop branch, a release-branch from develop should be created. This allows to freeze the state that should be released while features are still being merged to develop. The release-branch is typically tested again and then merged to master once everything is fine. Afterwards the master branch is merged back to the develop branch in order to make any changes made on the release branch also available in the develop branch.
  • hotfix/* If changes need to be applied onto the production code that cannot wait until develop is released again, a hotfix-branch can be used. A hotfix-branch is created from master and is merged back directly into master.

The following figure illustrates how the branches in the GitFlow-model are related to each other:

Image from alias

For more details on GitFlow please refer to this documentation (opens new window) and this cheatsheet (opens new window).

This section described the basic principles and terminology of Git at a high-level. For more details or informations about the many other features of the version-control software, please refer to the reference manual (opens new window) or the book Pro Git (opens new window) by Scott Chacon and Ben Straub that can be read online for free.

# Git in ONE DATA Apps

The previous section covered the basics of Git and discussed the most important features of the git-management software GitLab. This section will show how to use the Git-integration of ONE DATA Apps in combination with GitLab.

# Set an App Git Managed

ONE DATA Apps features basic versioning out-of-the-box by saving versions of the App's configuration that can be accessed through the App history. With the Git-integration it is possible to leverage the advantages and possibilities of Git during App development. This requires to enable Git-management for the App. All configurations created starting at that point will then be reflected in a Git repository as a commit and support for branching will be available in the App Builder.

To enable Git-management for an App, admin rights for that App are needed. For details about the Access roles in Apps please confer to the documentation about Access roles. Also, it is necessary that one of management tools for Git (GitLab or Gitea) is set up and the ONE DATA instance used is configured to use it. If it is not, the option to set the App Git-managed will not be shown.

The admin can open the App to be set git-managed in the App Builder and go to the App Settings (1) via the left sidebar. There the admin can choose to manage the App with Git (2).

Image from alias

When activating the toggle for the first time, a project with the name of the App will be created at the connected instance of the Git management tool. The master-branch will be initialized with the App configuration that is currently published to the App Viewer. The latest valid configuration will be pushed to the develop-branch. Please make sure that the configuration is valid to prevent losing changes when activating Git-management.

Via that toggle, it is also possible to remove the Git-management from the App again. In this case, the git-support will not be available anymore and subsequent modifications to the App's configuration will not be reflected in the repository. When re-enabling Git-management, the App will be connected to the already existing repository again, and the latest published/valid configuration will be pushed to master/develop respectively.

When setting an App to be Git-managed, the users having access to it and their Access roles will be reflected in the GitLab project or Gitea repository. Please note that only users given access to the App via the User Access option are added as members to the corresponding project or repository. Users having access to the App via Group Access or Global Access will not be granted any access. The members are assigned a certain permission level based on their role in the App. Please refer here to explore more about roles mapping in GitLab and here for Gitea.

Gitea Usage

Please coordinate the Gitea set-up and usage with a ONE LOGIC contact person, as Gitea is currently supported in an experimental mode.

Gitea App management

Currently, Gitea does not support OAuth2 and automatic creation of users. Before setting an App Git-manage it is necessary to create Gitea accounts for the users who are given user-access to an App including the one who sets an App to be Git-managed. For Account login, an e-mail address should be used to access the ONE DATA instance.

In case an OAuth2 authentication provider is configured (see here) the GitLab user accounts for ONE DATA users are identified by their linked ONE DATA ID. Otherwise, users are identified by their e-mail address. If a user who has access to the a App does not have a GitLab account, a new account is created with their user data. If an OAuth authentication provider for ONE DATA is configured, the newly created user is also linked to their ONE DATA account.

# Accessing GitLab and Gitea instances

# Logging into GitLab

If the GitLab server is configured to use OAuth2 authentication with ONE DATA, it is possible to log in to GitLab with ONE DATA credentials. In this case, a respective option is shown on the login page of GitLab. Image from alias

To log in with ONE DATA credentials choose OneData (1) in the section Sign in with instead of using the login form above. If the instance does not support the OAuth2 login, a GitLab admin needs to create an account with the same e-mail address used in ONE DATA. In this case, the login has to be done with the login form at the top.

# Logging into Gitea

As mentioned above, to access a Gitea instance one has to firstly register there using their ONE DATA login. Then it will be possible to log in to a Gitea instance using the aforementioned login and selected password.

# Structure of App repositories

The repository of a Git-managed App contains the App's configuration split up into multiple JSON-files. The directory structure reflects the structure of the JSON in the RAW-tab of the App Builder.

Image from alias

Arrays like datasources, elements, etc. that have Elements with an id are represented as separate JSON-files in the respective folder. The name of these files is always their ID, e.g., elements/welcomeTitle.json. Elements of the App configuration that do not have an ID (e.g., version and screens) are put as JSON-files into the root-directory. Apart from that, there is also a _index.json at the root of the repository. This file is used by ONE DATA Apps internally and should not be modified manually.
If the configuration is not valid JSON, it can't be split up. In this case, the raw JSON will be stored in invalid.json.

# Working with branches in the App Builder

When working with a Git-managed App, branching support provided by the App Builder can be used. When opening a Git-managed App, a dropdownon top of the file-tree will be visible. It is also shown in the History tab accessible via the left sidebar.

Image from alias

In the branch-dropdown the currently checked out branch (1) is visible. The editor shows the App configuration in the version of this branch. Furthermore, the configuration history shows only the history of the checked-out branch. To change the branch click on the dots to open the dropdown. There a new branch (2) that originates from develop can be created. After choosing a name for the branch, the new branch is checked out by the App Builder. Changes to the App configuration will be reflected as commits to that branch. Please note that it is recommended to use the GitFlow naming scheme for branches, i.e., starting the branch-name with feature/ followed by the name of the feature. In the branch-dropdown exisiting branches can be searched and selected (3). When choosing a branch from the list, the App configuration and history will be adapted to that branch, and one can start working on it. Changes made on a branch can be explored by clicking the button (4) next to the branch name to open the GitLab or Gitea repository overview for that branch in a new tab.

# Making changes to a Git-managed App

After checking out a branch via the branch-dropdown is is possible to start making changes to the App's configuration with the editor of the App Builder as done normally in a non-git-managed App. Please note that users with Builder permission can only modify feature-branches. Only App admins can modify develop directly.

Whenever making changes to the App configuration in the App Builder and saving it, a commit on the checked-out branch is created. It will receive the version number (e.g., #123) as commit message. Please note that while it is possible to change the name of the configuration version in the App Builder, this will currently not reword the respective commit.

To also directly jump to an older commit of a branch the configuration history can be used. It only lists the commits that are on the current branch. Please note that the numbers assigned to the versions are chosen uniquely across all branches to prevent identical names once a branch is merged. This means that some numbers of the version can be missing in the history if a commit was made on another branch in the meantime.

When publishing a version of an App to the App Viewer, this will also be reflected in the Git repository. The master branch always contains the version of the App that is currently published to the App Viewer. So when publishing a version through the App Settings or the App History, the develop-branch up to that version will be merged into master. Please note that this is done via a force-push overwriting any changes that have been previously on master.

# Merging work into develop

Once done with the work on a feature-branch, a merge request can be created to integrate it back to develop. Open the repository of the App and select Merge Requests (GitLab) in the left sidebar or New Pull Request (Gitea). In GitLab a merge request can be created as described here, and in Gitea a pull request created as shown here. Once the merge/pull request is merged by the assignee, the changes are available at the target branch in the App Builder after a few seconds. If a merge commit was necessary to merge the changes, this will also show up as a configuration version with the name of the MR/PR in the App History.

# Compare two Versions

Git allows to compare any revisions in the history of a project. This can be useful to check what has changed since a specific version or for example if when checking what parts of the code are affected by a specific feature. To compare two revisions the Compare page in GitLab that is accessible via the Repository menu in the left sidebar can be used. On this page source revision that will be used as the basis for the diff can be selected. The target is the revision the should be compared the source revision to. For the source and target any branch, tag, or commit can be entered. A commit can be chosen by entering its commit hash in the input field of the respective dropdown (please note that Enter has the be pressed afterwards).

Image from alias

Once clicking Compare a comparison similar to the one in merge requests is shown. At the top the commits that were made to get from the source revision to the target revision are visible. Below the changes between the versions are shown. The red lines exist in the source version, but not in the target version. The green lines are present in the target version, but not in the source version.

# Making changes to an App from a local repository

The repository of an App can also be cloned to make changes locally. This can be useful for example to resolve complicated conflicts with any preferred merge tool. The repository can be cloned as described by the steps here for GitLab and here for Gitea. Now it is possible to make changes in a local copy of the repository, like resolving the merge conflicts. Afterwards, with the following commands, a commit can be created and the changes pushed. Alternatively, a tool like SourceTree (opens new window) can be used.

git add --all
git commit -m "The commit message"
git push origin feature/the-branch

Once the changes are pushed, a configuration version with the commit message as the name should appear in the App Builder within a few seconds.