Communicating with API from CI/CD (GitLab)

[patrickgilsf]

I have a nodeJS script that connects my company's netbox site to a Google Sheet that I use as a front end for an internal app. It works when I run on my computer, but when its automated in a pipeline, it won't connect. I'm wondering if its because its not an allowed host. Can anybody lead me to the right path here? Do I need to make a request from our system admin about something? Thanks.

[candlerb]

Check what HTTP status code is returned to the CI/CD request, and what is in the body. That will probably tell you all you need to know. If Gitlab doesn't record this info (and it would be broken if it doesn't) then at worst you can use tcpdump on the Netbox server to capture it:

tcpdump -i lo -nn -s0 -A tcp port 8001

... although you'll have all your other Netbox activity mixed in.

"Allowed hosts" is the list of server names that the Netbox server can be accessed as. That is, if your server is 1.2.3.4, and in the DNS you've mapped netbox.example.com to 1.2.3.4, and ALLOWED_HOSTS is "netbox.example.com", it means that clients are allowed to make requests to http[s]://netbox.example.com/ but not http[s]://1.2.3.4/.

[patrickgilsf]

Thanks for your response. So, I'm not getting an http status code, because the script isn't connecting to the server at all, as far as I can tell. I'm getting an 'ERROR: connect ETIMED ipaddress:443. Don't think its a Gitlab thing, as I've had other requests to other APIs, on other scripts, that HAVE given me http status codes... like a 429 if I'm requesting too much data too many times. This particular one won't connect at all though. And, as I said, this same script does communicate successfully with the netbox api when it's run from my computer, and not the pipeline.

tcpdump -i lo -nn -s0 -A tcp port 8001f

this looks like a pcap that I can run from terminal? Never used it before. What I would I be looking for, if I were to be monitoring packets and trying to catch the error?

I should mention that I am authenticating with a token I generated on my netbox account. The credentials for this account are linked to our SSO, and I'm not totally sure how that process works either. If my SSO PW changes, so does my netbox one, similar to all of our SSO services.

My suspicion was that the token is not the only acceptance criteria for netbox to allow rest api communication, and it had something to do with the ip address that GitLab is coming from, vs my computer, which is on our VPN. So I was wondering if perhaps there is a way I can allow the Gitlab pipeline's IP address or network domain, from Netbox's perspective, or maybe there is just some small permission (like a checkbox) needed on the admin side of Netbox. Its a bit above my head, but I do have a handful of other scripts, without having any issues connecting to different APIs. Just something blocking this transmission, which I can only guess that the other APIs I communicate with don't care where the request is coming from, as long as the credentials are correct.

Any further help would be greatly appreciated.

[candlerb]

ERROR: connect ETIMED ipaddress:443

That says that Gitlab is failing even to establish a TCP connection to Netbox. It could mean that some firewall is blocking traffic from the Gitlab's IP address - maybe a network device in between, maybe iptables on the Netbox server.

If you have ssh access to the Gitlab server, then you can login there and try using curl to access the Netbox server. If it fails, then talk to your networking people about fixing it.

If you have ssh access to the Netbox server, then you can try this:

tcpdump -i eth0 -nn 'tcp port 443 and host x.x.x.x'

where you change eth0 to your external interface name (use ip addr list to find it), and x.x.x.x is the IP address of your gitlab server. Do you see nothing when the Gitlab CI/CD runs? Then it's 100% sure that either the gitlab server isn't making a request, or that some firewall in between is blocking the traffic.

If traffic is arriving then you can try another tcpdump, to see if the HTTPS traffic (now decrypted) is being proxied to Netbox:

tcpdump -i lo -nn -s0 -A tcp port 8001

But at this point it seems unlikely that it's getting this far.

[patrickgilsf]

After thinking about this further, I think my mistake here is that I am trying to connect a publicly hosted service (Gitlab pipeline) to a tool hosted on a private server. I actually hadn't realized that Netbox was just a private tool on our servers, and wasn't a public site. I was confused bc it has such an extensive rest API. I don't use it all that often. So, obviously when GitLab requests the restAPI, it doesn't have any resolution for the domain name, and doesn't know where to go. Honestly, I'm surprised that Netbox has an http API that is so robust, considering its only accessed within private domains. I'll have to figure out what people do in these situations normally.

[candlerb]

You can:

• Configure your network so that Netbox can be accessed from the outside world (e.g. via a reverse proxy on the border)
  • In your case only /api/ needs proxying in; you can add extra authentication on top of the token if you wish.
• Host your own Netbox instance in the cloud somewhere
• Use Netbox Cloud
• Host GitLab internally

[patrickgilsf]

Hey thanks...I was actually able to add an nc command to the yml file, and see if it could reach out to netbox.....and we found out that netbox lived in a IP different space than our (internally hosted) gitlab workers were. Which is something to change. Infrastructure team is fixing that. Much appreciate for your help here.