Home → Problems →

Unable to connect to web.archive.org on Ubuntu images

• Edit

DDoSolitary

02 Sep, 2018 06:34 AM

appveyor@av-0209-060743-294nqt84bw8tt9jk:~$ nslookup web.archive.org
Server:         169.254.169.254
Address:        169.254.169.254#53

Non-authoritative answer:
web.archive.org canonical name = wwwb-front3.us.archive.org.
Name:   wwwb-front3.us.archive.org
Address: 207.241.225.186

appveyor@av-0209-060743-294nqt84bw8tt9jk:~$ curl -L https://web.archive.org/web/20150802061047/http://support.wdc.com/download/archive/pciscsi.exe -v
*   Trying 207.241.225.186...
* connect to 207.241.225.186 port 443 failed: Connection timed out
* Failed to connect to web.archive.org port 443: Connection timed out
* Closing connection 0
curl: (7) Failed to connect to web.archive.org port 443: Connection timed out

1. Support Staff 1 Posted by Owen McDonnell on 04 Sep, 2018 09:20 PM

   

   I haven't seen any problem on other curl commands in Ubuntu builds. Are you able to run this exact curl command on a local Ubuntu machine?

   • Edit

2. 2 Posted by DDoSolitary on 05 Sep, 2018 04:09 AM

   

   Yes, other curl commands work well, the timeout error only occurs when trying to connect to web.archive.org

   Output when running on a Vultr VPS:

   [root@vultr ~]# curl -L https://web.archive.org/web/20150802061047/http://support.wdc.com/download/archive/pciscsi.exe -v > /dev/null
     % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                    Dload  Upload   Total   Spent    Left  Speed
     0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 207.241.225.186...
   * TCP_NODELAY set
   * Connected to web.archive.org (207.241.225.186) port 443 (#0)
   * ALPN, offering h2
   * ALPN, offering http/1.1
   * successfully set certificate verify locations:
   *   CAfile: /etc/ssl/certs/ca-certificates.crt
     CApath: none
   } [5 bytes data]
   * TLSv1.2 (OUT), TLS handshake, Client hello (1):
   } [213 bytes data]
   * TLSv1.2 (IN), TLS handshake, Server hello (2):
   { [102 bytes data]
   * TLSv1.2 (IN), TLS handshake, Certificate (11):
   { [4765 bytes data]
   * TLSv1.2 (IN), TLS handshake, Server key exchange (12):
   { [333 bytes data]
   * TLSv1.2 (IN), TLS handshake, Server finished (14):
   { [4 bytes data]
   * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
   } [70 bytes data]
   * TLSv1.2 (OUT), TLS change cipher, Client hello (1):
   } [1 bytes data]
   * TLSv1.2 (OUT), TLS handshake, Finished (20):
   } [16 bytes data]
   * TLSv1.2 (IN), TLS handshake, Finished (20):
   { [16 bytes data]
   * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
   * ALPN, server accepted to use h2
   * Server certificate:
   *  subject: OU=Domain Control Validated; CN=*.archive.org
   *  start date: Dec 19 20:59:01 2016 GMT
   *  expire date: Feb 21 22:56:08 2020 GMT
   *  subjectAltName: host "web.archive.org" matched cert's "*.archive.org"
   *  issuer: C=US; ST=Arizona; L=Scottsdale; O=GoDaddy.com, Inc.; OU=http://certs.godaddy.com/repository/; CN=Go Daddy Secure Certificate Authority - G2
   *  SSL certificate verify ok.
   * Using HTTP2, server supports multi-use
   * Connection state changed (HTTP/2 confirmed)
   * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
   } [5 bytes data]
   * Using Stream ID: 1 (easy handle 0x55d0d4b77ff0)
   } [5 bytes data]
   > GET /web/20150802061047/http://support.wdc.com/download/archive/pciscsi.exe HTTP/2
   > Host: web.archive.org
   > User-Agent: curl/7.61.0
   > Accept: */*
   >
   { [5 bytes data]
   * Connection state changed (MAX_CONCURRENT_STREAMS == 10)!
   } [5 bytes data]
     0     0    0     0    0     0      0      0 --:--:--  0:00:04 --:--:--     0< HTTP/2 200
   < server: nginx/1.13.11
   < date: Wed, 05 Sep 2018 04:07:20 GMT
   < content-type: application/octet-stream
   < x-archive-orig-x-cnection: close
   < x-archive-orig-x-powered-by: ASP.NET
   < x-archive-orig-set-cookie: BIGipServersupport_wdc_com.app~support_wdc_com_pool=3440378890.20480.0000; path=/, TS012ad883=01c2bc90a15cac7f6f44f2deaf23aaefd128f662be2e222d70353119b00432ab4e6708b4bd0b60c1c4c665d68fd44e0811b9ffbc4c; Path=/
   < x-archive-orig-accept-ranges: bytes
   < x-archive-orig-content-length: 510478
   < x-archive-orig-last-modified: Fri, 15 Feb 2002 22:44:00 GMT
   < x-archive-orig-etag: "0386c4272b6c11:0"
   < x-archive-orig-date: Sun, 02 Aug 2015 06:10:44 GMT
   < cache-control: max-age=1800
   < x-archive-guessed-content-type: application/x-dosexec
   < memento-datetime: Sun, 02 Aug 2015 06:10:47 GMT
   < link: <http://support.wdc.com/download/archive/pciscsi.exe>; rel="original", <https://web.archive.org/web/timemap/link/http://support.wdc.com/download/archive/pciscsi.exe>; rel="timemap"; type="application/link-format", <https://web.archive.org/web/http://support.wdc.com/download/archive/pciscsi.exe>; rel="timegate", <https://web.archive.org/web/20020607031935/http://support.wdc.com:80/download/archive/pciscsi.exe>; rel="first memento"; datetime="Fri, 07 Jun 2002 03:19:35 GMT", <https://web.archive.org/web/20070803180841/http://support.wdc.com/download/archive/pciscsi.exe>; rel="prev memento"; datetime="Fri, 03 Aug 2007 18:08:41 GMT", <https://web.archive.org/web/20150802061047/http://support.wdc.com/download/archive/pciscsi.exe>; rel="memento"; datetime="Sun, 02 Aug 2015 06:10:47 GMT", <https://web.archive.org/web/20180829133333/http://support.wdc.com/download/archive/pciscsi.exe>; rel="next memento"; datetime="Wed, 29 Aug 2018 13:33:33 GMT", <https://web.archive.org/web/20180829133335/https://support.wdc.com/download/archive/pciscsi.exe/>; rel="last memento"; datetime="Wed, 29 Aug 2018 13:33:35 GMT"
   < content-security-policy: default-src 'self' 'unsafe-eval' 'unsafe-inline' data: blob: archive.org web.archive.org analytics.archive.org
   < x-archive-src: archiveteam_archivebot_go_20150802120002/aur.archlinux.org-inf-20150801-012832-5w33j-00019.warc.gz
   < x-app-server: wwwb-app103
   < x-ts: ----
   < x-location: All
   < x-cache-key: httpsweb.archive.org/web/20150802061047/http://support.wdc.com/download/archive/pciscsi.exeJP
   < x-page-cache: MISS
   <
   { [14652 bytes data]
   100  498k    0  498k    0     0  97345      0 --:--:--  0:00:05 --:--:--  134k
   * Connection #0 to host web.archive.org left intact
   

   • Edit

3. 3 Posted by DDoSolitary on 05 Sep, 2018 04:17 AM

   

   MTR reports generated on the build server

   appveyor@av-0509-040503-3jhacx5dr6ylshpk:~$ mtr -rw web.archive.org
   Start: Wed Sep  5 04:13:32 2018
   HOST: av-0509-040503-3jhacx5dr6ylshpk          Loss%   Snt   Last   Avg  Best  Wrst StDev
     1.|-- router-central1-b.c.appveyor-ci.internal  0.0%    10    0.3   0.4   0.2   1.7   0.0
     2.|-- 216.239.58.214                            0.0%    10   36.5  36.7  36.5  37.8   0.0
     3.|-- 108.170.242.249                           0.0%    10   36.5  36.9  36.4  38.0   0.5
     4.|-- ???                                      100.0    10    0.0   0.0   0.0   0.0   0.0
     5.|-- cr1-200p-b-hu-0-7-0-21-0.bb.as11404.net   0.0%    10   37.9  38.1  37.9  38.8   0.0
     6.|-- agg2-200p-a-be-5.as11404.net              0.0%    10   38.0  38.1  37.8  38.7   0.0
     7.|-- archive.org-BE.demarc.spectrumnet.us      0.0%    10   37.8  37.9  37.7  38.9   0.0
     8.|-- ???                                      100.0    10    0.0   0.0   0.0   0.0   0.0
   
   appveyor@av-0509-040503-3jhacx5dr6ylshpk:~$ mtr -rw --tcp -P 443 web.archive.org
   Start: Wed Sep  5 04:14:35 2018
   HOST: av-0509-040503-3jhacx5dr6ylshpk          Loss%   Snt   Last   Avg  Best  Wrst StDev
     1.|-- router-central1-b.c.appveyor-ci.internal  0.0%    10    0.3   0.5   0.3   1.4   0.0
     2.|-- 72.14.237.146                             0.0%    10   37.1  36.8  36.2  37.9   0.0
     3.|-- 108.170.242.89                            0.0%    10   37.3  37.1  36.4  37.9   0.3
     4.|-- ???                                      100.0    10    0.0   0.0   0.0   0.0   0.0
     5.|-- cr1-200p-b-hu-0-7-0-21-0.bb.as11404.net   0.0%    10   38.0  38.2  37.9  39.0   0.0
     6.|-- agg2-200p-a-be-5.as11404.net              0.0%    10   38.1  38.0  37.7  38.6   0.0
     7.|-- archive.org-BE.demarc.spectrumnet.us      0.0%    10   37.8  37.9  37.8  38.4   0.0
     8.|-- ???
   

   • Edit

4. Support Staff 4 Posted by Wasa Pleshakov on 06 Sep, 2018 11:12 AM

   

   It looks like misconfiguration in one of our clouds. We trying to pin the root cause now.

   • Edit

5. 5 Posted by DDoSolitary on 19 Sep, 2018 02:44 AM

   

   Any updates on this? Thanks.

   • Edit

6. Support Staff 6 Posted by Ilya Finkelshte... on 20 Sep, 2018 08:29 PM

   

   Seems that they blacklisted public IPs of datacenter where Ubuntu VMs located. We wrote to them but got no answer so far. You can use some proxy server with curl to achieve your results.

   • Edit

7. Ilya Finkelshteyn closed this discussion on 20 Nov, 2018 09:00 PM.