<commitBuildModifications> does not update appear to update "latest commit" value (Bug #284)


Added by Tyler McGavran over 3 years ago. Updated over 3 years ago.


Status:Closed Start date:07/01/2013
Priority:Normal Due date:
Assignee:Tyler McGavran % Done:

0%

Category:Source Control Spent time: -
Target version:1.8.4
Affected version:1.8.3

Description

When using a Git Source Control block to tag and commit changes made during a build, CCNet does not appear to track these committed changes as the "latest" commit. As such, if one attempts to push these changes to the remote repository that the CCNet project is pointing to, the project will enter an endless cycle of building and pushing as each set of changes pushed is now "newer" than the "latest" commit tracked by CCNet.
Some sort of ability to push changes made during a build to a remote repository, or at least a fix to avoid this endless cycling of building and pushing would be appreciated.


History

Updated by Ruben Willems over 3 years ago

maybe the filtersource control can help you : Filtered

  • Status changed from New to Feedback
  • Assignee set to Tyler McGavran
  • Target version set to 1.8.4

Updated by Tyler McGavran over 3 years ago

Thank you for the response, but I am already using a filtered source control block. I am sadly one of those people who either give too much or too little information when explaining things, so bear with me.

I am an intern working helping a company migrate their source control to Git. They use CCNet and as such their build scripts need to change to accommodate the change to Git. Part of their build scripts updates the assembly versions of some files and they needs those updated numbers to be committed to the source repository.

CCNet can handle committing and tagging changes just fine, I am making good use of that option already. But, there is not built in "push back" feature in the Git source control block. If I manually make a NAnt target that only pushes the changes made by CCNet back to the source repository I run into a problem.

In the project's state file there are two variables of interest; the "LastChangeNumber" name/value pair and the <SourceControl> element (name=commit, value=SHA-1 ID). These numbers store the SHA-1 ID of the most recent commit in the source repository found when CCNet "git fetches" from the source repository. This number is NOT updated when CCNet commits and tags any changes made during the build. Now this makes sense; most people probably don't want to "push back" the results of a build, so why would the commit made by CCNet be considered the new "latest" commit. But if I make a NAnt target that manually pushes back this commit a funny problem arises.

The "push back" target (as I've called it) will successfully push the changes made by CCNet to the Git source repository. So far so good. But when CCNet comes looking around for some new changes, it doesn't report back "nothing new" like expected. That commit that was pushed has a different SHA-1 ID than what CCNet knows as the "latest" commit (found during the last build cycle). So, another build is triggered (there is a "newer" commit in the repository that what CCNet has), which pushes back the changes made, which triggers a new build. Wash, rinse, repeat.

I do have a solution running at this time, its just not a "beautiful" solution and has some small quirks to it. In short I output the SHA-1 of the commit by CCNet to a text file, manually update the values in the project's state file with that value and touch the ccnet.config file to ensure that the changes are acknowledged by CCNet. The fact that I have to "touch" the ccnet.config file leads to some small quirks here and there, but they are not relevant to this discussion.

I just think that it would be nice of the Git Source Control block gave a "push back" option that would allow someone to take the commit made by CCNet and push it back to the repository without causing an endless cycle of update, build, push, update, build, push... It would also be nice if there were a way to have CCNet acknowledge this without having to "touch" the ccnet.config file, as this causes its own set of problems

I realize this is a stupidly long response; if there is a better place to put explanations like these outside of Issues area, point me to it and I will stop taking up so much space here.

Updated by Ruben Willems over 3 years ago

So project A has a change, CCNet pulls the code and compiles. Updating the version and the like.
Now these assemblies go into the repo of project B, which is indeed a 'change'
You want this build of project B to be discarded?

Updated by Tyler McGavran over 3 years ago

Not quite. The assemblies go back to the repository of project A. Changes made during the build are pushed back to the source repository of the project. This would indeed causes a "change" in project A's repository. I would want the next build cycle to acknowledge that this "change" was the result of the previous build cycle, and therefore no new build need be started.

As I mentioned I have a solution in place, I just don't like it too much. I was just looking for a more elegant solution.

Updated by Ruben Willems over 3 years ago

Just a question : why publish those binaries under the same root as the source?
Is a separate project / folder not more meaningfull?
As a developper I do not really care about the binaries of the source when I do a 'get latest'
this would make the repo a lot bigger over time

suppose you have the following layout :
ProjectX/Main/Dev/Source
ProjectX/Main/Dev/Lib
ProjectX/Main/Release

you place the assemblies in ProjectX/Main/Release
and CCNet monitors ProjectX/Main/Dev
I think this should also fix it with 1 repo.

But I'll see what I can do to 'fix' this more elegantly
I'm no git expert...

Updated by Tyler McGavran over 3 years ago

This setup does make sense and I can mention it to my superiors, but I'm just the intern.
I am just trying to get it setup the way they want it and the way they had it in the old
system (using Borland's StarTeam) they pushed changes back to the source repository.

I misused some vocabulary earlier on I believe, so I'll correct myself now.
They don't push back binaries or assemblies made by the build (like exe's or dll's).
They use a third party program to update the version numbers in several files.
Its those changed version numbers that I am trying to push back to the source.
Resulting assemblies (like exe's and dll's) are copied to a directory elsewhere.

I think I might have been mistaken in placing this under the "Bug" section of the issues;
I apologize for any confusion or hindrance I am causing. It would be better to say that in
the future it would be helpful to have a pre-made option in the source control block to do
this, rather than have to create my own.

I'll post my current solution separately.
Thank you for all of your time given to helping me already, it is greatly appreciated.

Updated by Tyler McGavran over 3 years ago

<target name="PushBack" depends="init">
    <exec program="git.exe" commandline="checkout <TAG NAME>-${CCNetLabel}" />
    <!--
    <TAG NAME> is just part of the tag that I setup in the Git source control block. The tag has the
        format of <TAG NAME>-<BUILD LABEL>, and as such this has to be hard coded to be checked out properly.
    -->

    <exec program="git.exe" commandline="branch -f ${Git.branch} HEAD" />
    <!--
    ${Git.branch} is a previously defined variable, its just the name of the branch that the CCNet project
        is building. It can be hard coded if desired.

    After the checkout "branch -f" will forcibly move a branch to the SHA1-ID in question. Since HEAD points
        to the most recent checkout (now the latest tag), this forcibly "updates" the branch being built to
        the latest commit.
    -->

    <exec program="git.exe" commandline="rev-parse HEAD" output="${CCNetWorkingDirectory}/LatestChangeset.txt" />
    <!--
    "rev-parse" will return the SHA1-ID of the object given, in this case the latest tag made by CCNet
         (HEAD points to the current checkout, the latest tag in this case)

    Since I am using NAnt, I have to direct the output of "rev-parse" to a file (no output to property option)
    -->

    <loadfile
        file="${CCNetWorkingDirectory}/LatestChangeset.txt" 
        property="Temp" />
    <property name="LatestChangeset" value="${string::trim-end(Temp)}" />

    <xmlpoke
        file="C:/Program Files (x86)/CruiseControl.NET/server/${CCNetProject}.state" 
        xpath="/IntegrationResult/SourceControl[@name = 'commit']/@value" 
        value="${LatestChangeset}" />
    <if test="${not string::contains(CCNetBuildCondition, 'Force')}">
        <xmlpoke
            file="C:/Program Files (x86)/CruiseControl.NET/server/${CCNetProject}.state" 
            xpath="/IntegrationResult/Parameters/NameValuePair[@name = '$LastChangeNumber']/@value" 
            value="${LatestChangeset}" />
    </if>
    <!--
    I then load the value in the file to a property and replace the relevant values in the project's .STATE file
        with that property.

    LastChangeNumber only exists if latest build was not a "ForceBuild", so it can only be updated under similar
        conditions
    -->

    <exec program="git.exe" commandline="push origin HEAD:${Git.branch}" />
    <!--
    I then push back to the source repository (always origin when using CCNet)
    -->

    <touch file="C:/Program Files (x86)/CruiseControl.NET/server/ccnet.config" />
    <!--
    Finally, I have to "touch" the config file to ensure that the changes to the .STATE file
        are acknowledged; they are not watched by CCNet, so changes made to them are not
        recognized until CCNet reloads the config file.
    -->
</target>

Updated by Ruben Willems over 3 years ago

Sounds like the change is not needed :-)
You have the source in the repo, let a build system get, compile, test, whatever with it.
When all is ok, you label/tag the source with a version number.
There is no need to check in the changed files!
Because the will be overwritten / updated after each build.

I do not know the languange of the programs you use, but updating a version number is very simple.
This can easily be done via Nant or msbuild.
The projects I work on are all in Visual Studio, and I have 1 real version file.
All projects link to this 1 file : Add existing item, add as link (change the value of the combo-box)

You can take a look at the CCNet source, there the same setup is done.

Updated by Tyler McGavran over 3 years ago

I think I see where there is a disconnect with CCNet does version numbers and how I'm using them.

What you say makes sense, but the company I work for has their own process. They also use Visual Studio and have AssemblyInfo files; however, instead of overwriting the version number during the build time they simply increment it. They use a program called UpdateVersion that CAN replace the version number, but they use it to simply INCREMENT the number. Because of this if the changed version number is not pushed back to the source repository, the number will be incremented to the same value every build cycle.

But that is not your problem to fix. Once again, thank you very much for your time and help. I'll just stick with I have for now, no need to fix what isn't broken.

Updated by Ruben Willems over 3 years ago

  • Status changed from Feedback to Closed

Also available in: Atom PDF