Skip to content

All should buld the docker images (and therefore the binaries and protos via dependancies), and not the exes directly. #1399

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
May 20, 2019
Merged

All should buld the docker images (and therefore the binaries and protos via dependancies), and not the exes directly. #1399

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
ebcd2d0
All should buld the docker images (and therefore the binaries and pro…
tomwilkie May 18, 2019
5097444
Commit the generated protos so Cortex can be used as a dependancy.
tomwilkie May 18, 2019
cc141c3
Check protos are consistent as part of CI.
tomwilkie May 20, 2019
d092b20
Make sure we track all protos
gouthamve May 20, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,9 @@ jobs:
- run:
name: Check vendor directory is consistent.
command: make BUILD_IN_CONTAINER=false mod-check
- run:
name: Check protos are consistent.
command: make BUILD_IN_CONTAINER=false check-protos

test:
<<: *defaults
Expand Down
4 changes: 0 additions & 4 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,4 @@ cmd/cortex/cortex
.uptodate
.pkg
.cache
pkg/ingester/client/cortex.pb.go
pkg/querier/frontend/frontend.pb.go
pkg/ring/ring.pb.go
pkg/chunk/storage/caching_index_client.pb.go
images/
32 changes: 20 additions & 12 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.PHONY: all test clean images protos exes generated
.PHONY: all test clean images protos exes
.DEFAULT_GOAL := all

# Boiler plate for bulding Docker containers.
Expand All @@ -9,10 +9,11 @@ GIT_REVISION := $(shell git rev-parse HEAD)
UPTODATE := .uptodate

# Building Docker images is now automated. The convention is every directory
# with a Dockerfile in it builds an image calls quay.io/weaveworks/<dirname>.
# with a Dockerfile in it builds an image calls quay.io/cortexproject/<dirname>.
# Dependencies (i.e. things that go in the image) still need to be explicitly
# declared.
%/$(UPTODATE): %/Dockerfile
@echo
Copy link
Contributor

@gouthamve gouthamve May 20, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's this change for?

Copy link
Contributor Author

@tomwilkie tomwilkie May 20, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I put some new lines in the output of the makefile to make it easier to follow what it was doing. I like them, but not super attached. WDYT?

Copy link
Contributor

@gouthamve gouthamve May 20, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, thanks! LGTM!!!!

$(SUDO) docker build --build-arg=revision=$(GIT_REVISION) -t $(IMAGE_PREFIX)$(shell basename $(@D)) $(@D)/
$(SUDO) docker tag $(IMAGE_PREFIX)$(shell basename $(@D)) $(IMAGE_PREFIX)$(shell basename $(@D)):$(IMAGE_TAG)
touch $@
Expand Down Expand Up @@ -40,7 +41,7 @@ MAIN_GO := $(shell find . $(DONT_FIND) -type f -name 'main.go' -print)
EXES := $(foreach exe, $(patsubst ./cmd/%/main.go, %, $(MAIN_GO)), ./cmd/$(exe)/$(exe))
GO_FILES := $(shell find . $(DONT_FIND) -name cmd -prune -o -name '*.pb.go' -prune -o -type f -name '*.go' -print)
define dep_exe
$(1): $(dir $(1))/main.go $(GO_FILES) generated
$(1): $(dir $(1))/main.go $(GO_FILES) protos
$(dir $(1))$(UPTODATE): $(1)
endef
$(foreach exe, $(EXES), $(eval $(call dep_exe, $(exe))))
Expand All @@ -50,14 +51,13 @@ pkg/ingester/client/cortex.pb.go: pkg/ingester/client/cortex.proto
pkg/ring/ring.pb.go: pkg/ring/ring.proto
pkg/querier/frontend/frontend.pb.go: pkg/querier/frontend/frontend.proto
pkg/chunk/storage/caching_index_client.pb.go: pkg/chunk/storage/caching_index_client.proto
all: exes $(UPTODATE_FILES)
pkg/distributor/ha_tracker.pb.go: pkg/distributor/ha_tracker.proto
all: $(UPTODATE_FILES)
test: $(PROTO_GOS)
protos: $(PROTO_GOS)
mod-check: protos
configs-integration-test: $(PROTO_GOS)
lint: $(PROTO_GOS)

# And now what goes into each image
build-image/$(UPTODATE): build-image/*

# All the boiler plate for building golang follows:
Expand All @@ -83,10 +83,12 @@ NETGO_CHECK = @strings $@ | grep cgo_stub\\\.go >/dev/null || { \

ifeq ($(BUILD_IN_CONTAINER),true)

exes $(EXES) generated $(PROTO_GOS) lint test shell mod-check: build-image/$(UPTODATE)
exes $(EXES) protos $(PROTO_GOS) lint test shell mod-check check-protos: build-image/$(UPTODATE)
@mkdir -p $(shell pwd)/.pkg
@mkdir -p $(shell pwd)/.cache
$(SUDO) time docker run $(RM) $(TTY) -i \
@echo
Copy link
Contributor

@gouthamve gouthamve May 20, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here.

@echo ">>>> Entering build container: $@"
@$(SUDO) time docker run $(RM) $(TTY) -i \
-v $(shell pwd)/.cache:/go/cache \
-v $(shell pwd)/.pkg:/go/pkg \
-v $(shell pwd):/go/src/github.com/cortexproject/cortex \
Expand All @@ -96,7 +98,9 @@ configs-integration-test: build-image/$(UPTODATE)
@mkdir -p $(shell pwd)/.pkg
@mkdir -p $(shell pwd)/.cache
DB_CONTAINER="$$(docker run -d -e 'POSTGRES_DB=configs_test' postgres:9.6)"; \
$(SUDO) docker run $(RM) $(TTY) -i \
@echo
Copy link
Contributor

@gouthamve gouthamve May 20, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here.

@echo ">>>> Entering build container: $@"
@$(SUDO) docker run $(RM) $(TTY) -i \
-v $(shell pwd)/.cache:/go/cache \
-v $(shell pwd)/.pkg:/go/pkg \
-v $(shell pwd):/go/src/github.com/cortexproject/cortex \
Expand All @@ -111,8 +115,6 @@ configs-integration-test: build-image/$(UPTODATE)

else

generated: $(PROTO_GOS) $(MIGRATION_DIRS)

exes: $(EXES)

$(EXES):
Expand Down Expand Up @@ -145,13 +147,19 @@ mod-check:
GO111MODULE=on go mod vendor
@git diff --exit-code -- go.sum go.mod vendor/

check-protos: clean-protos protos
@git diff --exit-code -- $(PROTO_GOS)

endif

clean:
$(SUDO) docker rmi $(IMAGE_NAMES) >/dev/null 2>&1 || true
rm -rf $(UPTODATE_FILES) $(EXES) $(PROTO_GOS) .cache
rm -rf $(UPTODATE_FILES) $(EXES) .cache
go clean ./...

clean-protos:
rm -rf $(PROTO_GOS)

save-images:
@mkdir -p images
for image_name in $(IMAGE_NAMES); do \
Expand Down
1 change: 0 additions & 1 deletion docs/getting_started.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ Cortex can also make use of external memcacheds for caching and although these a
For simplicity & to get started, we'll run it as a single process with no dependencies:

```sh
$ make protos # Build all the protobuffer definitions.
$ go build ./cmd/cortex
$ ./cortex -config.file=./cmd/cortex/single-process-config.yaml
```
Expand Down
Loading