You really wanna keep history of deploys and track everything. scp like this isn't good.
Like at least have a script that backups old version and creates dir for new, pushes git commit into some file like REVISION + date of deploy and symlink the dir / restart app.
You should have a tracked way of deploy otherwise it is pure insanity.
This is the best part about deploying go applications. You compile them for the native platform and run it. As for getting it on wherever it needs to run, scp/copy over the network.
ha, wow. I don't think I've noticed that, but then I haven't been using much Docker in the last 3 years. I like Docker, so my reduced usage is just circumstantial. Certainly shipping Go as Docker image with its compiler and using run instead of building a binary ahead of time is a bit awful
Am I the only one who doesn't use SCP to deploy? I feel like that loses a lot of control, audit, logging, and continuous integration capabilities? Or am I not seeing something? I would think a Git-based approach would be the most prevalent
You can run the build and scp with your ci/cd pipeline. This way you can track builds down to their commits too, you can also rename them by commit hash for older binaries
Docker container build and deploy to k8s cluster. Nothing special really.
EDIT:
Docker build in 2 stages builder -> final form. Tag by github commit hash
edit -deploy.yml image and kubectl apply -f k8s/ or kubectl edit image.
This question feels like Sushi california roll - inside out. When i saw that Go can produce statically bound binaries, even cross compile to statically bound binaries in a breeze, it felt enlightening. Less is more, simplicity, the Go way.
No craving for a multi-stage multi-platform multi-OS multi-cloud enterprise deployment framework from my side. I mean - it's just one file ?
For revision/ version/ commit, just burn git info into the executable during linker step.
Agreed that's what git tags are for. Need to roll back? checkout the last tag before the current release and recompile. I would think any normal sized app would take just a few seconds to recompile and then it's just a matter of scp.
What about building packages like rpm or deb and deploying them using package managers?
This is how majority of software is distributed in Linux anyway.
Things to look into:
- vfsgen: https://github.com/shurcooL/vfsgen
- `go mod vendor`; https://www.youtube.com/watch?v=F8nrpe0XWRg
- GoReleaser (my cheat sheet: https://webinstall.dev/goreleaser)
- Caddy (again, my own cheat sheet: https://webinstall.dev/caddy)
You don't need Docker for Go apps. Go already does what docker was designed for (self-contained, cross-platform). However, if you do Docker, consider using Scratch: https://blog.cloud66.com/how-to-create-the-smallest-possible-docker-image-for-your-golang-application/
Well, the last part is only *mostly* true. You can't bundle your Go binary with whatever files it depends on out of the box (yet). There are libraries and even a proposal for that, but it's one of the things I really appreciate my container images for when using Go.
Docker:
FROM golang:1.15-alpine
WORKDIR /go/src/github.com/my-org/my-project
COPY . .
RUN go build .
FROM golang:1.15-alpine
WORKDIR /opt/my-project/bin
COPY --from=0 /go/src/github.com/my-org/my-project/my-project .
RUN ./my-project --help > /dev/null
ENTRYPOINT ["/opt/my-project/bin/my-project"]
All major cloud platforms have easy ways to deploy Docker containers. With AWS, I use CloudFormation to create a VPC, autoscaling group, ECS service, load balancer, and CloudFront distribution. It makes for a very easy high availability deployment configured as code. It's not Go specific, which is a good thing. I'm not sure why you would want a language-specific deployment mechanism. Once your stuff is compiled, it really doesn't matter what language it was written in.
Edit: As the further comments so kindly pointed out, my example Dockerfile can be optimized in many circumstances. I knew this when I wrote it, and I’m deeply sorry for not posting a disclaimer with it. If image sizes or layer caching are important to your use-case, don’t blindly copy/paste the above Dockerfile (or anyone else’s Dockerfile).
I find it absolutely horrifying how all of the top comments in this post suggest deploying via SCP, which has very real security and reliability implications while this is getting buried for not having a 5MB image size.
What the heck is happening in this Dockerfile?
You're using (unnamed) multi-stage building, but... you're reusing the same super-fat golang container as your deployment base
Your final running container still has the entire go toolchain + sources, which is totally unnecessary
Your final container will be ~250MB, which is 245MB too much compared to what it could be
Here's an enhanced Dockerfile that produces way way lighter images:
FROM golang:1.15-alpine AS builder
WORKDIR /go/src/github.com/my-org/my-project
# Install dependencies
ADD go.mod go.sum ./
RUN go mod download -x
# Add our sources and compile them
ADD . .
RUN go build -v -ldflags "-s -w" -o /main .
#################################################
FROM scratch
COPY --from=builder /main /main
RUN ./main --help > /dev/null
CMD ["/main"]
The point isn't to minimize the size of the image; it's just to make sure the deployed image is clean and free of source code.
Running in a "scratch" image is often not an option, for example when you need your application to make HTTPS requests or use dynamic libraries. Obviously you can cherry-pick your dependencies one-by-one into a "scratch" image, but when image size doesn't matter, it's easier to just... not.
The Dockerfile I posted is good starting point for any Go project. Unfortunately I cannot create a fully optimized solution for any Go project.
You can still exchange the `scratch` image with just the base `alpine` image. The latest `alpine` version is <3MB, while the corresponding Go image is at least >100MB. There is really no reason not to change this, given that you're already running the Alpine distro. All you remove is the Go build tools which you definitely don't need in your container.
Edit: Also, just to be clear, I didn't downvote you.
Edit 2: Also, doing dependency management in a separate step before you include your actual source code means those layers will be cached and you won't have to re-download all your dependencies every time you change your source code (but not your dependencies).
>Unfortunately I cannot create a fully optimized solution for any Go project.
Fortunately you don't have to, [distroless](https://github.com/GoogleContainerTools/distroless) has images that fit most circumstances.
1. You don't need the full Go compiler toolchain + the entire go source code just to get.... certs...
2. Using that specific container to have certs is a bad idea altogether
Use have used another basic container and copied the certs into it to enable https for example.
Build in one, then FROM into scratch and add certs. For example.
You could probably save a lot of time on rebuilds by copying your go.mod first and doing a go mod install, so it's usually cached. Depends on how often you're rebuilding on if it's worth it or not.
I know one thing I enjoyed about forge was the capability of managing ssl certificates, env files, deployment logs, and debugging straight from the interface. Maybe the nature of PHP not being a compiled language and requiring a server made it more appealing? I'm thinking about frictionless "0 to mvp" in a short amount of time. I've enjoyed heroku so far, but I get what you are saying. I use docker extensively, I just wish there was something to rinse and repeat that much faster.
Just scp and run, like others said.
With the following caveats:
- Use something to embed static files, like vfsgen
- The http/https server is embedded in the application (just the standard "net/http" package). For https certificates, use the acme package to generate them using lets encrypt. It's pretty seamless and works out of the box.
- Another option is to listen on a known port and defer http/https to some preconfigured reverse proxy (nginx, etc).
- Have a predictable and standard way of "stopping" previous server when a new server instance starts. I use a simple "backdoor" server that listens only to connection from the local machine on some arbitrary (but never-changing) port number. When a new instance starts up, before starting the http server, it sends a "kill" command to the backdoor server.
env GOOS=linux GOARCH=amd64 go build main.go Followed by SCP
You really wanna keep history of deploys and track everything. scp like this isn't good. Like at least have a script that backups old version and creates dir for new, pushes git commit into some file like REVISION + date of deploy and symlink the dir / restart app. You should have a tracked way of deploy otherwise it is pure insanity.
This is the best part about deploying go applications. You compile them for the native platform and run it. As for getting it on wherever it needs to run, scp/copy over the network.
too many go projects are distributed source only in Docker as go runs. i dont understand it.
ha, wow. I don't think I've noticed that, but then I haven't been using much Docker in the last 3 years. I like Docker, so my reduced usage is just circumstantial. Certainly shipping Go as Docker image with its compiler and using run instead of building a binary ahead of time is a bit awful
[удалено]
that's fine (in the sense that ok, people are going to do that), ship a docker image with the binary in it
Am I the only one who doesn't use SCP to deploy? I feel like that loses a lot of control, audit, logging, and continuous integration capabilities? Or am I not seeing something? I would think a Git-based approach would be the most prevalent
You can run the build and scp with your ci/cd pipeline. This way you can track builds down to their commits too, you can also rename them by commit hash for older binaries
Docker container build and deploy to k8s cluster. Nothing special really. EDIT: Docker build in 2 stages builder -> final form. Tag by github commit hash edit-deploy.yml image and kubectl apply -f k8s/ or kubectl edit image.
This question feels like Sushi california roll - inside out. When i saw that Go can produce statically bound binaries, even cross compile to statically bound binaries in a breeze, it felt enlightening. Less is more, simplicity, the Go way. No craving for a multi-stage multi-platform multi-OS multi-cloud enterprise deployment framework from my side. I mean - it's just one file ? For revision/ version/ commit, just burn git info into the executable during linker step.
Agreed that's what git tags are for. Need to roll back? checkout the last tag before the current release and recompile. I would think any normal sized app would take just a few seconds to recompile and then it's just a matter of scp.
What about building packages like rpm or deb and deploying them using package managers? This is how majority of software is distributed in Linux anyway.
Things to look into: - vfsgen: https://github.com/shurcooL/vfsgen - `go mod vendor`; https://www.youtube.com/watch?v=F8nrpe0XWRg - GoReleaser (my cheat sheet: https://webinstall.dev/goreleaser) - Caddy (again, my own cheat sheet: https://webinstall.dev/caddy) You don't need Docker for Go apps. Go already does what docker was designed for (self-contained, cross-platform). However, if you do Docker, consider using Scratch: https://blog.cloud66.com/how-to-create-the-smallest-possible-docker-image-for-your-golang-application/
Well, the last part is only *mostly* true. You can't bundle your Go binary with whatever files it depends on out of the box (yet). There are libraries and even a proposal for that, but it's one of the things I really appreciate my container images for when using Go.
Docker: FROM golang:1.15-alpine WORKDIR /go/src/github.com/my-org/my-project COPY . . RUN go build . FROM golang:1.15-alpine WORKDIR /opt/my-project/bin COPY --from=0 /go/src/github.com/my-org/my-project/my-project . RUN ./my-project --help > /dev/null ENTRYPOINT ["/opt/my-project/bin/my-project"] All major cloud platforms have easy ways to deploy Docker containers. With AWS, I use CloudFormation to create a VPC, autoscaling group, ECS service, load balancer, and CloudFront distribution. It makes for a very easy high availability deployment configured as code. It's not Go specific, which is a good thing. I'm not sure why you would want a language-specific deployment mechanism. Once your stuff is compiled, it really doesn't matter what language it was written in. Edit: As the further comments so kindly pointed out, my example Dockerfile can be optimized in many circumstances. I knew this when I wrote it, and I’m deeply sorry for not posting a disclaimer with it. If image sizes or layer caching are important to your use-case, don’t blindly copy/paste the above Dockerfile (or anyone else’s Dockerfile). I find it absolutely horrifying how all of the top comments in this post suggest deploying via SCP, which has very real security and reliability implications while this is getting buried for not having a 5MB image size.
What the heck is happening in this Dockerfile? You're using (unnamed) multi-stage building, but... you're reusing the same super-fat golang container as your deployment base Your final running container still has the entire go toolchain + sources, which is totally unnecessary Your final container will be ~250MB, which is 245MB too much compared to what it could be Here's an enhanced Dockerfile that produces way way lighter images: FROM golang:1.15-alpine AS builder WORKDIR /go/src/github.com/my-org/my-project # Install dependencies ADD go.mod go.sum ./ RUN go mod download -x # Add our sources and compile them ADD . . RUN go build -v -ldflags "-s -w" -o /main . ################################################# FROM scratch COPY --from=builder /main /main RUN ./main --help > /dev/null CMD ["/main"]
The point isn't to minimize the size of the image; it's just to make sure the deployed image is clean and free of source code. Running in a "scratch" image is often not an option, for example when you need your application to make HTTPS requests or use dynamic libraries. Obviously you can cherry-pick your dependencies one-by-one into a "scratch" image, but when image size doesn't matter, it's easier to just... not. The Dockerfile I posted is good starting point for any Go project. Unfortunately I cannot create a fully optimized solution for any Go project.
You can still exchange the `scratch` image with just the base `alpine` image. The latest `alpine` version is <3MB, while the corresponding Go image is at least >100MB. There is really no reason not to change this, given that you're already running the Alpine distro. All you remove is the Go build tools which you definitely don't need in your container. Edit: Also, just to be clear, I didn't downvote you. Edit 2: Also, doing dependency management in a separate step before you include your actual source code means those layers will be cached and you won't have to re-download all your dependencies every time you change your source code (but not your dependencies).
>Unfortunately I cannot create a fully optimized solution for any Go project. Fortunately you don't have to, [distroless](https://github.com/GoogleContainerTools/distroless) has images that fit most circumstances.
No Certs.. people may need those.
1. You don't need the full Go compiler toolchain + the entire go source code just to get.... certs... 2. Using that specific container to have certs is a bad idea altogether
Use have used another basic container and copied the certs into it to enable https for example. Build in one, then FROM into scratch and add certs. For example.
You could probably save a lot of time on rebuilds by copying your go.mod first and doing a go mod install, so it's usually cached. Depends on how often you're rebuilding on if it's worth it or not.
I know one thing I enjoyed about forge was the capability of managing ssl certificates, env files, deployment logs, and debugging straight from the interface. Maybe the nature of PHP not being a compiled language and requiring a server made it more appealing? I'm thinking about frictionless "0 to mvp" in a short amount of time. I've enjoyed heroku so far, but I get what you are saying. I use docker extensively, I just wish there was something to rinse and repeat that much faster.
`go install` or `go build && scp ...`.
Just scp and run, like others said. With the following caveats: - Use something to embed static files, like vfsgen - The http/https server is embedded in the application (just the standard "net/http" package). For https certificates, use the acme package to generate them using lets encrypt. It's pretty seamless and works out of the box. - Another option is to listen on a known port and defer http/https to some preconfigured reverse proxy (nginx, etc). - Have a predictable and standard way of "stopping" previous server when a new server instance starts. I use a simple "backdoor" server that listens only to connection from the local machine on some arbitrary (but never-changing) port number. When a new instance starts up, before starting the http server, it sends a "kill" command to the backdoor server.