Use Makefile to simplify setup and commands

[MonkeyCanCode]

This PR brings in a root-level Makefile to simplify and centralize how we set up our development environment and run common build tasks for Polaris.

The Problem I Noticed:
Right now, getting set up or performing routine builds often means jumping between different documentation pages to find various tool installations (like helm, kubectl, jq, jenv, git, docker) and then copying lengthy gradlew commands. While it makes sense to have granular instructions for deep dives or specific contributions, having a single, straightforward entry point for common tasks could really smooth out onboarding for new folks and make our development flow much more efficient.

My Proposal (and a Working Example):
I've put together this PR as a first pass, a working example, to kick off a discussion on whether this centralized Makefile approach is something we want to adopt. It shows how we can:

• Streamline Dependency Setup: Get all common project dependencies installed with just one make command. (We can definitely refine this further, perhaps breaking it down into component-specific sets if we build this out.)
• Simplify Builds: Replace those long gradlew calls with concise make commands to build the server, admin, or both.
• Automate Helm Tasks: Easily generate Helm documentation, run unit tests, and perform linting all with simple make targets.
• Add Pre-commit Automation: Introduce a pre-commit target to automate routine tasks like Helm documentation generation and applying Spotless formatting.
  • Heads up: Python codebase linting/checks aren't included yet. I hit some hard-coded path issues there, but that's something I can tackle if we decide to pursue this direction.
• Minikube Cluster Management: Streamlining cluster setup, teardown, and interaction specifically for Helm-related development.

Beyond this PR: Other Potential Use Cases:
If we move forward with this approach, the Makefile could also become a central point for:

• Unified Test Execution: Running all project test cases (unit, integration, etc.) with a single make command.
• Service State Management & Deployment: Handling local service lifecycle (e.g., make start, make stop) and orchestrating Helm deployments.

What I'm Looking For:
I'd really appreciate your thoughts on this centralized Makefile idea. Does this feel like the right path for how we want to manage our developer experience? Any feedback on the overall concept or the specific examples here would be super helpful as we decide whether to build this out further.

Mailing discussion:
https://lists.apache.org/thread/vjnptg3prdps9slkt7rnvv7ldcwld88s

Sample interface and commands output:
Main interface:

➜  polaris git:(makefile) make

Usage:
  make <target>

General
  help                            Display this help.

Polaris Build
  build                           Build Polaris server, admin, and container images
  build-server                    Build Polaris server and container image
  build-admin                     Build Polaris admin and container image
  build-cleanup                   Clean build artifacts
  spotless-apply                  Apply code formatting using Spotless Gradle plugin.

Helm
  helm-doc-generate               Generate Helm chart documentation
  helm-unittest                   Run Helm chart unittest
  helm-lint                       Run Helm chart lint check

Minikube
  minikube-start-cluster          Start the Minikube cluster.
  minikube-stop-cluster           Stop the Minikube cluster.
  minikube-load-images            Load local Docker images into the Minikube cluster.
  minikube-cleanup                Clean up and delete the Minikube cluster.

Pre-commit
  pre-commit                      Run tasks for pre-commit

Dependencies
  setup-dependencies              Install required binaries if not present

Run pre-commit

➜  polaris git:(makefile) make pre-commit
--- Checking Homebrew installation ---
--- Homebrew is installed ---
--- Checking and installing required binaries ---
--- All required binaries checked/installed ---
--- Applying Spotless formatting ---
Configuration on demand is an incubating feature.

[Incubating] Problems report is available at: file:///Users/yong/Desktop/GitHome/polaris/build/reports/problems/problems-report.html

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

For more on this, please refer to https://docs.gradle.org/8.14.2/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.

BUILD SUCCESSFUL in 1s
174 actionable tasks: 174 up-to-date
--- Spotless formatting applied ---
--- Generating Helm documentation ---
INFO[2025-07-11T00:19:38-05:00] Found Chart directories [polaris]
INFO[2025-07-11T00:19:38-05:00] Generating README Documentation for chart helm/polaris
--- Helm documentation generated and copied ---

Run build

➜  polaris git:(makefile) make build
--- Checking Homebrew installation ---
--- Homebrew is installed ---
--- Checking and installing required binaries ---
--- All required binaries checked/installed ---
--- Building Polaris server ---
Configuration on demand is an incubating feature.

[Incubating] Problems report is available at: file:///Users/yong/Desktop/GitHome/polaris/build/reports/problems/problems-report.html

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

For more on this, please refer to https://docs.gradle.org/8.14.2/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.

BUILD SUCCESSFUL in 16s
85 actionable tasks: 2 executed, 83 up-to-date
--- Polaris server build complete ---
--- Building Polaris admin ---
Configuration on demand is an incubating feature.

[Incubating] Problems report is available at: file:///Users/yong/Desktop/GitHome/polaris/build/reports/problems/problems-report.html

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

For more on this, please refer to https://docs.gradle.org/8.14.2/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.

BUILD SUCCESSFUL in 7s
67 actionable tasks: 2 executed, 65 up-to-date
--- Polaris admin build complete ---

Minikube state management

➜  polaris git:(makefile) make minikube-load-images
--- Checking Homebrew installation ---
--- Homebrew is installed ---
--- Checking and installing required dependencies for this target ---
--- All required dependencies checked/installed ---
--- Checking Minikube cluster status ---
--- Minikube cluster is already running. Skipping start ---
--- Loading images into Minikube cluster ---
--- Checking Homebrew installation ---
--- Homebrew is installed ---
--- Checking and installing required dependencies for this target ---
--- All required dependencies checked/installed ---
--- Building Polaris server ---
Configuration on demand is an incubating feature.

[Incubating] Problems report is available at: file:///Users/yong/Desktop/GitHome/polaris/build/reports/problems/problems-report.html

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

For more on this, please refer to https://docs.gradle.org/8.14.2/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.

BUILD SUCCESSFUL in 26s
85 actionable tasks: 2 executed, 83 up-to-date
--- Polaris server build complete ---
--- Building Polaris admin ---
Configuration on demand is an incubating feature.

[Incubating] Problems report is available at: file:///Users/yong/Desktop/GitHome/polaris/build/reports/problems/problems-report.html

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

For more on this, please refer to https://docs.gradle.org/8.14.2/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.

BUILD SUCCESSFUL in 9s
67 actionable tasks: 2 executed, 65 up-to-date
--- Polaris admin build complete ---
--- Images loaded into Minikube cluster ---

➜  polaris git:(makefile) minikube image ls
registry.k8s.io/pause:3.10
registry.k8s.io/kube-scheduler:v1.33.1
registry.k8s.io/kube-proxy:v1.33.1
registry.k8s.io/kube-controller-manager:v1.33.1
registry.k8s.io/kube-apiserver:v1.33.1
registry.k8s.io/etcd:3.5.21-0
registry.k8s.io/coredns/coredns:v1.12.0
gcr.io/k8s-minikube/storage-provisioner:v5
docker.io/apache/polaris:latest
docker.io/apache/polaris:1.1.0-incubating-SNAPSHOT
docker.io/apache/polaris-admin-tool:latest
docker.io/apache/polaris-admin-tool:1.1.0-incubating-SNAPSHOT

[snazy]

The script is likely a nice addition for macOS users who are using brew.
I noticed that it refers to Docker itself, which, IIRC requires users to have a paid license.

[MonkeyCanCode]

The script is likely a nice addition for macOS users who are using brew.
I noticed that it refers to Docker itself, which, IIRC requires users to have a paid license.

So we can switch to podman if that is a concern (or make docker substitute-able via env variable). Docker is still free for individuals or even small companies. However, most of the docs we have currently are using docker as opposed to podman. And yes, as you called out, this will be great for Linux/Unix users (brew can also be installed on WSL for Windows users). This doesn't work for native Windows users who are not using WSL (most of our docs are not Windows friendly at the moment as well). What do you think?

[dimas-b]

If users/developers find this Makefile helpful, I think we can certainly add it. However, I believe it would be preferable to keep Gradle as the primary / official entry point for local builds and CI.

[MonkeyCanCode]

If users/developers find this Makefile helpful, I think we can certainly add it. However, I believe it would be preferable to keep Gradle as the primary / official entry point for local builds and CI.

For sure, we don't necessary need to change the doc to replace the ones with gradlew commands. The main purpose for this one is to easy the process for developers who are not familiar with those tools and run 1-2 short commands to get things going. I will be adding more use cases (such as minikube) later this week. Again, thanks for your feedback and feel free to drop me things that you think can be benefits for developers to perform without copying/paste length commands from various pages.

[snazy]

Don't get me wrong, but it seems the setup is quite opinionated. The Makefile requires devs to use new things, that do not necessarily match their daily workflows. For example, I do not have brew on my Linux machine, as I do no need it. Many people use sdkman or rely on distributions' rpm or deb packages.

Certain things in this change however look useful. For example the helm stuff, so I wouldn't mind to have that as a separate "helper Makefile", but without the requirement for brew, jenv and a specific Java distribution.

Minikube can just work, but people may have special settings. I recall a bunch of "specialties" wrt minikube + podman + images published to minikube's registry.

The line between "non-commercial OSS use" and "commercial use" of Docker (i.e. you work on something for your employer) is blurry.

I don't mind this script in general, if it's helpful for some users, but I'm not convinced that it's suitable for most users.

[MonkeyCanCode]

Don't get me wrong, but it seems the setup is quite opinionated. The Makefile requires devs to use new things, that do not necessarily match their daily workflows. For example, I do not have brew on my Linux machine, as I do no need it. Many people use sdkman or rely on distributions' rpm or deb packages.

Certain things in this change however look useful. For example the helm stuff, so I wouldn't mind to have that as a separate "helper Makefile", but without the requirement for brew, jenv and a specific Java distribution.

Minikube can just work, but people may have special settings. I recall a bunch of "specialties" wrt minikube + podman + images published to minikube's registry.

The line between "non-commercial OSS use" and "commercial use" of Docker (i.e. you work on something for your employer) is blurry.

I don't mind this script in general, if it's helpful for some users, but I'm not convinced that it's suitable for most users.

I think the intension here is not to promo everyone to use brew but rather a quick command for people to get their setup ready if they have brew. As you pointed out, there are other package managers out there and people have their own preference of the tooling management. We can add other option to support additional installation methods (apk/yum/apt etc.). Also, the Makefile does checks if users have the tools installed, installation will be skipped if the needed tools are already present. In this case, if a user doesn't want to install brew and we only support brew, the makefile will just aborted said brew is missing etc. thus unable to fulfill the additional needed dependencies. Then to go around this, users will then be responsible to setup the needed dependencies to be able to use the Makefile until we supported their ideal package managements (e.g. some people really like to use golang to put tools under a given directory...that is a not a common package management but some people do use this route. We can add support for those if that are more preferred).

And yes, with the one I pushed last night, I had defined the needed dependencies per target. This mean, if an user is only working on helm, it won't ask user to setup jenv etc nor specific java distribution.

Then for the licensing concern with Docker, I don't see a problem with switching to podman. However, I do think we should make it configurable as not everyone work on Polaris from a corporate perspective (also, docker itself is free, the one that required license is docker desktop which has the GUI. Our current doc is installing docker desktop which is not free for large corp. That being said, within a large corporate, an user can still installed docker on his/her Linux machine without GUI to avoid the licensing concern).

For the minikube lifecycle management, this is mainly from @adutra feedback earlier with run.sh script which was used for manage lifecycle of kind cluster. IMO, learning how to use a tool is not hard but it does takes time. Rather a simple start/stop/cleanup/load will be more human friendly for a person who is not familiar the tool of selection. For the specific settings around minikube, we can see if they can be generic enough to add to this workflow, however, based on our current published doc, those are not there. This doesn't mean it won't work out of box, instead, it will just mean for an end-users who have special settings needed around minikube won't be using the start option from the Makefile to bring up a Minikube as the one we currently have is really simple and taking default settings.

I created this tool mainly to easy my workflow with Mac when working with Polaris. But we can for sure extend it to make it suitable for more users which are not using Mac or not using brew.

[MonkeyCanCode]

Don't get me wrong, but it seems the setup is quite opinionated. The Makefile requires devs to use new things, that do not necessarily match their daily workflows. For example, I do not have brew on my Linux machine, as I do no need it. Many people use sdkman or rely on distributions' rpm or deb packages.
Certain things in this change however look useful. For example the helm stuff, so I wouldn't mind to have that as a separate "helper Makefile", but without the requirement for brew, jenv and a specific Java distribution.
Minikube can just work, but people may have special settings. I recall a bunch of "specialties" wrt minikube + podman + images published to minikube's registry.
The line between "non-commercial OSS use" and "commercial use" of Docker (i.e. you work on something for your employer) is blurry.
I don't mind this script in general, if it's helpful for some users, but I'm not convinced that it's suitable for most users.

I think the intension here is not to promo everyone to use brew but rather a quick command for people to get their setup ready if they have brew. As you pointed out, there are other package managers out there and people have their own preference of the tooling management. We can add other option to support additional installation methods (apk/yum/apt etc.). Also, the Makefile does checks if users have the tools installed, installation will be skipped if the needed tools are already present. In this case, if a user doesn't want to install brew and we only support brew, the makefile will just aborted said brew is missing etc. thus unable to fulfill the additional needed dependencies. Then to go around this, users will then be responsible to setup the needed dependencies to be able to use the Makefile until we supported their ideal package managements (e.g. some people really like to use golang to put tools under a given directory...that is a not a common package management but some people do use this route. We can add support for those if that are more preferred).

And yes, with the one I pushed last night, I had defined the needed dependencies per target. This mean, if an user is only working on helm, it won't ask user to setup jenv etc nor specific java distribution.

Then for the licensing concern with Docker, I don't see a problem with switching to podman. However, I do think we should make it configurable as not everyone work on Polaris from a corporate perspective (also, docker itself is free, the one that required license is docker desktop which has the GUI. Our current doc is installing docker desktop which is not free for large corp. That being said, within a large corporate, an user can still installed docker on his/her Linux machine without GUI to avoid the licensing concern).

For the minikube lifecycle management, this is mainly from @adutra feedback earlier with run.sh script which was used for manage lifecycle of kind cluster. IMO, learning how to use a tool is not hard but it does takes time. Rather a simple start/stop/cleanup/load will be more human friendly for a person who is not familiar the tool of selection. For the specific settings around minikube, we can see if they can be generic enough to add to this workflow, however, based on our current published doc, those are not there. This doesn't mean it won't work out of box, instead, it will just mean for an end-users who have special settings needed around minikube won't be using the start option from the Makefile to bring up a Minikube as the one we currently have is really simple and taking default settings.

I created this tool mainly to easy my workflow with Mac when working with Polaris. But we can for sure extend it to make it suitable for more users which are not using Mac or not using brew.

I had add podman support and brew optional with the latest push, here is the new CLI interface:

➜  polaris git:(makefile) make

Usage:
  make <target>

General
  help                                      Display this help.

Polaris Build
  build                                     Build Polaris server, admin, and container images
  build-server                              Build Polaris server and container image
  build-admin                               Build Polaris admin and container image
  build-spark-plugin-3.5-2.12               Build Spark plugin v3.5 with Scala v2.12
  build-spark-plugin-3.5-2.13               Build Spark plugin v3.5 with Scala v2.13
  build-cleanup                             Clean build artifacts
  spotless-apply                            Apply code formatting using Spotless Gradle plugin.

Helm
  helm-doc-generate                         Generate Helm chart documentation
  helm-unittest                             Run Helm chart unittest
  helm-lint                                 Run Helm chart lint check

Minikube
  minikube-start-cluster                    Start the Minikube cluster
  minikube-stop-cluster                     Stop the Minikube cluster
  minikube-load-images                      Load local Docker images into the Minikube cluster
  minikube-cleanup                          Cleanup the Minikube cluster

Pre-commit
  pre-commit                                Run tasks for pre-commit

Dependencies
  check-dependencies                        Check if all requested dependencies are present
  install-dependencies-brew                 Install dependencies if not present via Brew
  install-optional-dependencies-brew        Install optional dependencies if not present via Brew

Here is a sample command in case if u want to play around with podman:

# build images to local
CONTAINER_TOOL=podman make build
# build images to local and push to minikube
CONTAINER_TOOL=podman make minikube-load-images

There is one catch found with podman build, where podman will try to add default registry (in this case docker.io as prefix to the image):

docker.io/apache/polaris-admin-tool                 latest                     30d60d73d7ba  51 seconds ago  505 MB
docker.io/apache/polaris-admin-tool                 1.1.0-incubating-SNAPSHOT  30d60d73d7ba  51 seconds ago  505 MB

This is a good practice but it will break our existed workflow. I can look into more (either update chart to use docker.io as well as update docker build to include the registry or check if there is a way to even remove it in podman...quick check shows later is not possible).

[MonkeyCanCode]

Don't get me wrong, but it seems the setup is quite opinionated. The Makefile requires devs to use new things, that do not necessarily match their daily workflows. For example, I do not have brew on my Linux machine, as I do no need it. Many people use sdkman or rely on distributions' rpm or deb packages.
Certain things in this change however look useful. For example the helm stuff, so I wouldn't mind to have that as a separate "helper Makefile", but without the requirement for brew, jenv and a specific Java distribution.
Minikube can just work, but people may have special settings. I recall a bunch of "specialties" wrt minikube + podman + images published to minikube's registry.
The line between "non-commercial OSS use" and "commercial use" of Docker (i.e. you work on something for your employer) is blurry.
I don't mind this script in general, if it's helpful for some users, but I'm not convinced that it's suitable for most users.

I think the intension here is not to promo everyone to use brew but rather a quick command for people to get their setup ready if they have brew. As you pointed out, there are other package managers out there and people have their own preference of the tooling management. We can add other option to support additional installation methods (apk/yum/apt etc.). Also, the Makefile does checks if users have the tools installed, installation will be skipped if the needed tools are already present. In this case, if a user doesn't want to install brew and we only support brew, the makefile will just aborted said brew is missing etc. thus unable to fulfill the additional needed dependencies. Then to go around this, users will then be responsible to setup the needed dependencies to be able to use the Makefile until we supported their ideal package managements (e.g. some people really like to use golang to put tools under a given directory...that is a not a common package management but some people do use this route. We can add support for those if that are more preferred).
And yes, with the one I pushed last night, I had defined the needed dependencies per target. This mean, if an user is only working on helm, it won't ask user to setup jenv etc nor specific java distribution.
Then for the licensing concern with Docker, I don't see a problem with switching to podman. However, I do think we should make it configurable as not everyone work on Polaris from a corporate perspective (also, docker itself is free, the one that required license is docker desktop which has the GUI. Our current doc is installing docker desktop which is not free for large corp. That being said, within a large corporate, an user can still installed docker on his/her Linux machine without GUI to avoid the licensing concern).
For the minikube lifecycle management, this is mainly from @adutra feedback earlier with run.sh script which was used for manage lifecycle of kind cluster. IMO, learning how to use a tool is not hard but it does takes time. Rather a simple start/stop/cleanup/load will be more human friendly for a person who is not familiar the tool of selection. For the specific settings around minikube, we can see if they can be generic enough to add to this workflow, however, based on our current published doc, those are not there. This doesn't mean it won't work out of box, instead, it will just mean for an end-users who have special settings needed around minikube won't be using the start option from the Makefile to bring up a Minikube as the one we currently have is really simple and taking default settings.
I created this tool mainly to easy my workflow with Mac when working with Polaris. But we can for sure extend it to make it suitable for more users which are not using Mac or not using brew.

I had add podman support and brew optional with the latest push, here is the new CLI interface:

➜  polaris git:(makefile) make

Usage:
  make <target>

General
  help                                      Display this help.

Polaris Build
  build                                     Build Polaris server, admin, and container images
  build-server                              Build Polaris server and container image
  build-admin                               Build Polaris admin and container image
  build-spark-plugin-3.5-2.12               Build Spark plugin v3.5 with Scala v2.12
  build-spark-plugin-3.5-2.13               Build Spark plugin v3.5 with Scala v2.13
  build-cleanup                             Clean build artifacts
  spotless-apply                            Apply code formatting using Spotless Gradle plugin.

Helm
  helm-doc-generate                         Generate Helm chart documentation
  helm-unittest                             Run Helm chart unittest
  helm-lint                                 Run Helm chart lint check

Minikube
  minikube-start-cluster                    Start the Minikube cluster
  minikube-stop-cluster                     Stop the Minikube cluster
  minikube-load-images                      Load local Docker images into the Minikube cluster
  minikube-cleanup                          Cleanup the Minikube cluster

Pre-commit
  pre-commit                                Run tasks for pre-commit

Dependencies
  check-dependencies                        Check if all requested dependencies are present
  install-dependencies-brew                 Install dependencies if not present via Brew
  install-optional-dependencies-brew        Install optional dependencies if not present via Brew

Here is a sample command in case if u want to play around with podman:

# build images to local
CONTAINER_TOOL=podman make build
# build images to local and push to minikube
CONTAINER_TOOL=podman make minikube-load-images

There is one catch found with podman build, where podman will try to add default registry (in this case docker.io as prefix to the image):

docker.io/apache/polaris-admin-tool                 latest                     30d60d73d7ba  51 seconds ago  505 MB
docker.io/apache/polaris-admin-tool                 1.1.0-incubating-SNAPSHOT  30d60d73d7ba  51 seconds ago  505 MB

This is a good practice but it will break our existed workflow. I can look into more (either update chart to use docker.io as well as update docker build to include the registry or check if there is a way to even remove it in podman...quick check shows later is not possible).

For the image lookup issue, it is actually a bug in kubernetes/minikube#19396. So the above instructions are still correct.

[dimas-b]

@MonkeyCanCode : Would you mind adding README notes about its intended use and the relationship to Gradle?

[MonkeyCanCode]

@MonkeyCanCode : Would you mind adding README notes about its intended use and the relationship to Gradle?

Added. Thanks for review.

[dimas-b]

Thx, @MonkeyCanCode !

[MonkeyCanCode]

thanks for review @eric-maynard, @snazy , and @dimas-b . I will leave this one up for couple more days before merge it. Once merged (assuming we would like to proceed with this route), then i will move those testing commands into this top level Makefile as well for python client and python code base lint etc. @HonahX

[eric-maynard]

The makefile itself looks good -- I also think the addition to the README is helpful context.

I do wonder how many build systems we want to end up with -- we could do the same thing with a shell script launched from gradle, for example.

IMO the convenience we gain here is worth it, but perhaps we should update other docs to actually point users at the makefile, such as the getting-started docs.

@HonahX may also have thoughts on our growing use of make.

[MonkeyCanCode]

The makefile itself looks good -- I also think the addition to the README is helpful context.

I do wonder how many build systems we want to end up with -- we could do the same thing with a shell script launched from gradle, for example.

IMO the convenience we gain here is worth it, but perhaps we should update other docs to actually point users at the makefile, such as the getting-started docs.

@HonahX may also have thoughts on our growing use of make.

IMO, a Makefile acts as a smart central organizer, which is often better than relying only on Gradle or separate Bash scripts. While Gradle is great for Java development, it gets complicated when you try to make it do everything, like building Python packages or deploying Helm charts (do-able via some thrid-party plugins which may get EOL at any time or using plain CMD plugin then handle various stdout/stderr parsing etc). Doing these external tasks directly in Gradle may lead to complex and hard-to-manage code in Gradle. Also, if parts of the project, like Helm charts, move to their own separate locations, a custom Gradle setup for those parts would have to be copied or redone. A Makefile, being more general, makes automation flexible and reusable.

We could do many things with a Bash script. But Makefiles offer built-in features that Bash doesn't easily provide, like handling task dependencies and running tasks at the same time with parallel execution. Recreating these in a Bash script for a big project would be a lot of extra work.