I have been experimenting with pull request builds for Bitbucket repositories.

I found that AppVeyor’s pull request support is very sensible. The build actually performs a git merge matching what Bitbucket would do (if it is configured to do a merge commit (it is possible to configure Bitbucket to do --ff-only on merges among other things and I don’t think I see AppVeyor implementing this) and assuming the auto-merge matches what Bitbucket would do). This way, the build is performed on the hypothetical result of the merge instead of on the unmerged source branch. This tries to protect the user from situations where the source branch can succeed but the merge results would fail.

Also, these pull request builds will occur for the target repository whenever a pull request is made. This means that AppVeyor does not need to be set up to build commits on the source repository in a cross-repository pull request scenario. This is particularly useful for “one private repo” plans.

However, it appears that Bitbucket has no support for this. There is no concept of a pull request build status in the API except for an aggregation API which lists the all(?) of the build results from commits in the pull request. Bitbucket only lets you set the status of concrete, already-pushed commits themselves.

However, Bitbucket’s pull request GUI will show build statuses. The build statuses it shows are the ones associated with the last commit in the pull request. This means that, currently, the only way to get AppVeyor build results to show up on a Bitbucket pull request is to have AppVeyor set up on the repository and branch originating the pull request.

Getting Bitbucket’s pull request GUI to show the results of a pull request-initiated build has high value to me. And Bitbucket supports various branch restrictions including some which are meant to support use of CI such as AppVeyor. The options in question are:

• “Minimum number of successful builds for the last commit with no failed builds and no in progress builds”. This can be set to the number of distinct build keys from different external build systems that are integrated with Bitbucket. For a project integrating only with AppVeyor, I can set this to 1.
• “Allow automatic merge when builds pass”. This setting can be enabled to allow users to hit “Merge” before the build is complete which causes the PR to wait and automatically merge once the build is complete. This doesn’t make for a fully automated pull request experience, but it is one step toward that.

There is also one additional setting which does not show up in the documentation which I think makes a compromise possible. Note that I have attached a screenshot showing the “Edit a branch restriction” screen in Bitbucket demonstrating this additional setting:

• “Maximum number of commits behind destination branch”. Setting this to 0 means that the pull request can be resolved through a fast forward. If this check is passing, that means that the pull request can be satisfied with a fast forward. If the pull request is resolved with a fast forward, that means that file contents of the source commit will be identical to the file contents after performing the fast forward merge onto the target commit.

Is it possible for AppVeyor to support setting the build status of a pull request by setting the build status of the source commit? Is this something which used to work and then was broken by Bitbucket at some point? Is Bitbucket’s model just too broken to be worth supporting—e.g., AppVeyor’s access token might not even let it set a build status on the source repository due to ownership issues.

Given that AppVeyor supports pull request builds for Bitbucket, what is the use case for this if it isn’t something that Bitbucket supports? Is there a way to notify some external automation system beside Bitbucket that the PR build succeeded so that this external automation system can manipulate the PR by, say, checking off a checklist item or marking it as “reviewed by CI”?

I do need to do some searching around in Atlassian. I think Bitbucket should learn about PR-specific build statuses because that would be so useful. I have not looked too much. But I also do not expect Atlassian to move very quickly even if they do decide to support this eventually. So I am wondering if there is some direction or advice from the AppVeyor side of things on this.

Thanks!