From git, perform shallow clone

Chris Dembia's Avatar

Chris Dembia

20 May, 2014 11:21 PM

My repository is large (200 MB), so it takes a while to download. However, we only need the last commit to do the test. Travis uses a --depth=50 flag on the git clone command. Could appveyor do this as well?

Showing page 2 out of 2. View the first page

  1. 31 Posted by Chris Dembia on 14 Jul, 2014 02:40 AM

    Chris Dembia's Avatar

    Feodor, you said "I see. Currently builds run on "Small" Azure instances
    with one CPU core. Wondering how long would it take to run it on "Medium"
    instance with 2 cores..."

    Is this something we could try? That would be exciting!!

  2. Support Staff 32 Posted by Feodor Fitsner on 14 Jul, 2014 09:12 AM

    Feodor Fitsner's Avatar

    Hi Chris,

    Do you want a custom trial plan with Medium instances enabled? What's your account name?

  3. 33 Posted by Chris Dembia on 14 Jul, 2014 02:48 PM

    Chris Dembia's Avatar

    That would be great! chrisdembia (through GitHub).

  4. Support Staff 34 Posted by Feodor Fitsner on 14 Jul, 2014 02:57 PM

    Feodor Fitsner's Avatar

    I will switch your account from Free to a custom plan with 1 concurrent job on Medium instances. For how long would you need that trial plan?

  5. 35 Posted by Chris Dembia on 14 Jul, 2014 04:33 PM

    Chris Dembia's Avatar

    Could we have it for 15 days?

  6. Support Staff 36 Posted by Feodor Fitsner on 14 Jul, 2014 05:23 PM

    Feodor Fitsner's Avatar

    Done! You are on custom plan now.

  7. 37 Posted by Chris Dembia on 14 Jul, 2014 05:50 PM

    Chris Dembia's Avatar

    Excellent!

  8. 38 Posted by Chris Dembia on 14 Jul, 2014 06:39 PM

    Chris Dembia's Avatar

    Feodor, thanks for doing this. Our build gets twice as far now, but still
    does not complete in the allotted 30 minutes. I'd say our build was about
    50-60% complete. Our build process benefits greatly from parallelism; more
    cores would certainly help.

  9. Support Staff 39 Posted by Feodor Fitsner on 14 Jul, 2014 06:52 PM

    Feodor Fitsner's Avatar

    I've increased build timeout to 90 minutes.

  10. 40 Posted by Chris Dembia on 14 Jul, 2014 09:47 PM

    Chris Dembia's Avatar

    Thanks Feodor...our build and tests ended up taking 36 minutes.

  11. Support Staff 41 Posted by Feodor Fitsner on 15 Jul, 2014 05:26 AM

    Feodor Fitsner's Avatar

    Very well, thanks for the update!

  12. 42 Posted by Vinney on 15 Jul, 2014 07:33 PM

    Vinney's Avatar

    I know Mercurial doesn't have a shallow clone feature. Do you have any work arounds for Mercurial/BitBucket? As is, our clone operation takes about 7 minutes to complete.

  13. Support Staff 43 Posted by Feodor Fitsner on 16 Jul, 2014 05:03 AM

    Feodor Fitsner's Avatar

    It's possible to download specific commit as zip file. Could you try downloading repo as zip (from BitBucket UI) to see if it takes less than 7 minutes?

  14. 44 Posted by Vinney on 16 Jul, 2014 04:51 PM

    Vinney's Avatar

    I see how to go about downloading the zip file as described here but I'm not seeing how to bypass/prevent the clone step.

  15. Support Staff 45 Posted by Feodor Fitsner on 16 Jul, 2014 05:26 PM

    Feodor Fitsner's Avatar

    No, I mean if you click download repo as zip from BitBucket UI - does it take 7 minutes?

  16. 46 Posted by Vinney on 18 Jul, 2014 10:17 PM

    Vinney's Avatar

    My project's zip file on BitBucket is 17MB vs 1.06GB for the entire repo. It takes me somewhere around 20 seconds to download the zip from BitBucket. Granted, my internet connection is likely higher bandwidth than what we're experiencing on the VM. Still, downloading the zip rather than the a full clone operation seems like a desirable option. Is this something that is possible today? I know I can download the zip with PS but is there any way to bypass the clone operation without losing the hook into the repository?

  17. Support Staff 47 Posted by Feodor Fitsner on 19 Jul, 2014 05:34 AM

    Feodor Fitsner's Avatar

    I asked you to check that because I think we can implement "shallow clone" for BitBucket, like we did for GitHub, so AppVeyor doesn't use hg command, but just downloads zip and unpacks it.

    Thanks, I will give it a try and let you know about the results.

  18. Support Staff 48 Posted by Feodor Fitsner on 19 Jul, 2014 03:24 PM

    Feodor Fitsner's Avatar

    Great news! We've just deployed an update supporting "shallow clone" for BitBucket repositories: public and private, git and mercurial. Set "Fetch repository as zip archive" on General tab of project settings or add shallow_clone: true into appveyor.yml.

    Let me know how it works for your project!

  19. 49 Posted by Vinney on 20 Jul, 2014 03:44 PM

    Vinney's Avatar

    Works like a charm! We're working on lowering our build process footprint so that we might condense our total time to the allotted 30 minutes. This will be a huge help! Thanks for adding the feature so quickly!

  20. Ilya Finkelshteyn closed this discussion on 25 Aug, 2018 01:46 AM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac