Skip to content

Two Tier Grabdish #436

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 127 commits into from
Jun 13, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
127 commits
Select commit Hold shift + click to select a range
6cf93bc
Test 1
RichardExley Feb 11, 2022
15f7926
test2
RichardExley Feb 11, 2022
67af58f
test3
RichardExley Feb 11, 2022
2c6d62d
test3
RichardExley Feb 11, 2022
f900a80
test4
RichardExley Feb 11, 2022
48ddda5
test5
RichardExley Feb 11, 2022
9d9d462
test6
RichardExley Feb 11, 2022
a15a98b
test
RichardExley Feb 11, 2022
4a75451
test
RichardExley Feb 11, 2022
bd233f1
test
RichardExley Feb 11, 2022
f0a3a78
test
RichardExley Feb 11, 2022
286faab
test
RichardExley Feb 11, 2022
212a8c3
test
RichardExley Feb 14, 2022
80dfe5d
test
RichardExley Feb 15, 2022
aa91931
test
RichardExley Feb 15, 2022
9d2476d
test
RichardExley Feb 15, 2022
531e45d
test
RichardExley Feb 15, 2022
de9aab4
test
RichardExley Feb 16, 2022
3a53461
test
RichardExley Feb 17, 2022
e54a56e
test
RichardExley Feb 17, 2022
f5ea70c
test
RichardExley Feb 17, 2022
0fa6bcf
test
RichardExley Feb 17, 2022
d6b5ecd
test
RichardExley Feb 17, 2022
f4a10c3
test
RichardExley Feb 18, 2022
90ecb20
test
RichardExley Feb 18, 2022
5e01aff
test
RichardExley Feb 18, 2022
c1d17d5
test
RichardExley Feb 28, 2022
57d8472
test
RichardExley Feb 28, 2022
5b21f92
Merge remote-tracking branch 'upstream/main' into grabdish-in-db
RichardExley Feb 28, 2022
f5f16f8
test
RichardExley Mar 1, 2022
fc67c84
test
RichardExley Mar 1, 2022
a2d59d3
test
RichardExley Mar 1, 2022
c37fcf9
test
RichardExley Mar 1, 2022
19f07c6
test
RichardExley Mar 1, 2022
30a2fd8
test
RichardExley Mar 2, 2022
d0c9d36
test
RichardExley Mar 2, 2022
df8159e
test
RichardExley Mar 2, 2022
8e3dbd7
test
RichardExley Mar 2, 2022
bd3c376
test
RichardExley Mar 2, 2022
af86c8a
test
RichardExley Mar 2, 2022
3db19a6
test
RichardExley Mar 3, 2022
b87f4b8
test
RichardExley Mar 3, 2022
866d355
test
RichardExley Mar 3, 2022
41d2c9f
test
RichardExley Mar 3, 2022
ba5d466
test
RichardExley Mar 3, 2022
f0947e5
test
RichardExley Mar 3, 2022
44cf30d
test
RichardExley Mar 4, 2022
dbfa59c
test
RichardExley Mar 4, 2022
1d0611c
test
RichardExley Mar 4, 2022
b09c30d
test
RichardExley Mar 4, 2022
e72fe1e
test
RichardExley Mar 4, 2022
cfe754f
test
RichardExley Mar 7, 2022
aa4f361
test
RichardExley Mar 7, 2022
c4197fd
test
RichardExley Mar 9, 2022
19b206a
test
RichardExley Mar 9, 2022
9c1d93f
test
RichardExley Mar 9, 2022
e4c5152
test
RichardExley Mar 9, 2022
e4acd63
test
RichardExley Mar 10, 2022
cdd85e7
test
RichardExley Mar 10, 2022
bb77423
test
RichardExley Mar 10, 2022
c543ba9
test
RichardExley Mar 10, 2022
8686ce4
test
RichardExley Mar 10, 2022
060faa0
test
RichardExley Mar 10, 2022
9fb77f9
test
RichardExley Mar 10, 2022
7567877
test
RichardExley Mar 11, 2022
1579cd5
test
RichardExley Mar 11, 2022
3191473
test
RichardExley Mar 11, 2022
a29d976
test
RichardExley Mar 11, 2022
63c607d
test
RichardExley Mar 11, 2022
32dde61
test
RichardExley Mar 11, 2022
7c894ec
test
RichardExley Mar 11, 2022
f38ee3e
test
RichardExley Mar 15, 2022
1fcd9f2
test
RichardExley Mar 15, 2022
91407d4
test
RichardExley Mar 15, 2022
35b0fa5
test js loader
RichardExley Mar 15, 2022
7b28cae
test
RichardExley Mar 15, 2022
e0675c1
test
RichardExley Mar 16, 2022
96069f9
test
RichardExley Mar 16, 2022
6abc092
test
RichardExley Mar 16, 2022
6c583c3
test
RichardExley Mar 16, 2022
cb47914
test
RichardExley Mar 16, 2022
2c27e05
test
RichardExley Mar 17, 2022
a320371
test
RichardExley Mar 17, 2022
89228ab
test
RichardExley Mar 18, 2022
44f6a5d
test
RichardExley Mar 18, 2022
a037c48
test
RichardExley Mar 18, 2022
a13638b
test
RichardExley Mar 18, 2022
433697f
test
RichardExley Mar 18, 2022
7a4f7ba
test
RichardExley Mar 18, 2022
aa9799d
test
RichardExley Mar 19, 2022
f50ec6a
test
RichardExley Mar 19, 2022
dc02dfc
test
RichardExley Mar 19, 2022
1c552b5
test
RichardExley Mar 22, 2022
b1cbd0e
test
RichardExley Mar 23, 2022
7218b4b
Merge remote-tracking branch 'upstream/main' into grabdish-in-db
RichardExley Mar 23, 2022
fbaa7df
test
RichardExley Mar 23, 2022
e3c2298
test
RichardExley Mar 24, 2022
551e779
test
RichardExley Mar 24, 2022
651f94f
test
RichardExley Mar 25, 2022
e5e1b5c
test
RichardExley Mar 25, 2022
c239f72
test
RichardExley Mar 25, 2022
fe6ca5e
test
RichardExley Mar 25, 2022
5988b73
test
RichardExley Mar 25, 2022
b21def3
test
RichardExley Mar 31, 2022
ec07308
test
RichardExley Mar 31, 2022
c5ba6e8
test
RichardExley Apr 1, 2022
111ae72
test
RichardExley Apr 1, 2022
56ff6c4
test
RichardExley Apr 1, 2022
0348810
test
RichardExley Apr 1, 2022
f84e346
test
RichardExley Apr 1, 2022
27b5be5
Update setup_functions.env
RichardExley Apr 14, 2022
3fe2500
Merge remote-tracking branch 'upstream/main' into grabdish-in-db
RichardExley Apr 14, 2022
7376e5d
test
RichardExley Apr 20, 2022
2926009
test
RichardExley Apr 20, 2022
f4edec8
test
RichardExley Apr 20, 2022
f43be0e
Merge remote-tracking branch 'upstream/main' into grabdish-in-db
RichardExley Apr 20, 2022
1f023b8
test
RichardExley Apr 20, 2022
a2bf6c5
test
RichardExley Apr 21, 2022
c1ee431
test
RichardExley Apr 21, 2022
9f219db
test
RichardExley Apr 21, 2022
2a76852
test
RichardExley Apr 21, 2022
74781e5
test
RichardExley Apr 21, 2022
f703966
test
RichardExley May 3, 2022
234e0a9
test
RichardExley May 3, 2022
81b9646
Merge remote-tracking branch 'upstream/main' into grabdish-in-db
RichardExley Jun 1, 2022
8e0d04d
Merge remote-tracking branch 'upstream/main' into grabdish-in-db
RichardExley Jun 13, 2022
80fa0de
Revert Changes Outside of Workshop Folder
RichardExley Jun 13, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
2 changes: 1 addition & 1 deletion grabdish/config/db/common/apply/order-object-scripts.sql
Original file line number Diff line number Diff line change
Expand Up @@ -169,4 +169,4 @@ EXCEPTION

END;
/
show errors
show errors
2 changes: 1 addition & 1 deletion grabdish/docs/Provisioning.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ The code that provisions the Grabdish application is located in the grabdish/con
## Prerequisites

The following are required before provisioning Grabdish:
1. Kubernetes cluster with kuebctl configured
1. Kubernetes cluster with kubectl configured
2. One or two databases
3. An OCI object store bucket (ATP 2DB only)
4. get_secret bash function (available in infra/vault/folder)
Expand Down
69 changes: 69 additions & 0 deletions workshops/dcms-db/config/apply.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
#!/bin/bash
# Copyright (c) 2021 Oracle and/or its affiliates.
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl.


# Fail on error
set -eu

if ! provisioning-helper-pre-apply; then
exit 1
fi

QUEUE_TYPE=$(state_get QUEUE_TYPE)
OCI_REGION="$(state_get OCI_REGION)"

# Generate the ssh keys
if ! test -d $MY_STATE/ssh; then
mkdir -p $MY_STATE/ssh
ssh-keygen -t rsa -N "" -b 2048 -C "db" -f $MY_STATE/ssh/dcmsdb
state_set SSH_PUBLIC_KEY_FILE $"$MY_STATE/ssh/dcmsdb.pub"
state_set SSH_PRIVATE_KEY_FILE "$MY_STATE/ssh/dcmsdb"
fi

# Copy terraform to my state
if ! test -d $MY_STATE/terraform; then
rm -rf $MY_STATE/terraform
cp -r $MSDD_WORKSHOP_CODE/$DCMS_WORKSHOP/config/terraform $MY_STATE
fi

# Start the provisioning apply
cd $MY_STATE/terraform
source terraform-env.sh

if ! terraform init; then
echo 'ERROR: terraform init failed!'
exit 1
fi

if ! terraform apply -auto-approve; then
echo 'ERROR: terraform apply failed!'
exit 1
fi

# Get the load balancer public IP
state_set LB_ADDRESS `terraform output -raw lb_address`

# Get the ORDS instance public IP
state_set ORDS_ADDRESS `terraform output -raw ords_address`

# Get the ORDS instance public IP
state_set DB_OCID `terraform output -raw db_ocid`

state_set TNS_ADMIN_ZIP_FILE $MY_STATE/terraform/uploads/adb_wallet.zip
TNS_ADMIN=$MY_STATE/tns_admin
mkdir -p $TNS_ADMIN
unzip $(state_get TNS_ADMIN_ZIP_FILE) -d $TNS_ADMIN
cat >$TNS_ADMIN/sqlnet.ora <<- !
WALLET_LOCATION = (SOURCE = (METHOD = file) (METHOD_DATA = (DIRECTORY="$TNS_ADMIN")))
SSL_SERVER_DN_MATCH=yes
!

state_set TNS_ADMIN $TNS_ADMIN

# Write the output
cat >$OUTPUT_FILE <<!
export LB_ADDRESS='$(state_get LB_ADDRESS)'
export ORDS_ADDRESS='$(state_get ORDS_ADDRESS)'
export TNS_ADMIN='$(state_get TNS_ADMIN)'
!
32 changes: 32 additions & 0 deletions workshops/dcms-db/config/destroy.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
#!/bin/bash
# Copyright (c) 2021 Oracle and/or its affiliates.
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl.

# Fail on error
set -e


if ! provisioning-helper-pre-destroy; then
exit 1
fi

cd $MY_STATE/terraform

# Workaround for issue where terraform fails in OCI cloud shell after a few days
rm -rf .terraform
rm -f .terraform.lock.hcl

# Start the provisioning destroy
source terraform-env.sh

if ! terraform init; then
echo 'ERROR: terraform init failed!'
exit 1
fi

if ! terraform destroy -auto-approve; then
echo 'ERROR: terraform apply failed!'
exit 1
fi

rm -f $STATE_FILE
7 changes: 7 additions & 0 deletions workshops/dcms-db/config/requirements.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/bin/bash
# Copyright (c) 2021 Oracle and/or its affiliates.
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl.

REQ_UTILS="touch rm mkdir"
REQ_INPUT_PARAMS=""
REQ_OUTPUT_PARAMS="LB_ADDRESS ORDS_ADDRESS TNS_ADMIN"
23 changes: 23 additions & 0 deletions workshops/dcms-db/config/terraform/availability_domain.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Copyright © 2020, Oracle and/or its affiliates.
# All rights reserved. The Universal Permissive License (UPL), Version 1.0 as shown at http://oss.oracle.com/licenses/upl

data "oci_identity_availability_domains" "availability_domains" {
compartment_id = var.tenancy_ocid
}

// If this is ALF, need to determine which AD can create CI's in
data "oci_limits_limit_values" "limits_limit_values" {
compartment_id = var.tenancy_ocid
service_name = "compute"
scope_type = "AD"
name = "vm-standard-e2-1-micro-count"
filter {
name = "value"
values = ["2"]
}
}

// If we have a value from limits, use that as ALF, otherwise use AD-1
locals {
availability_domain = length(data.oci_limits_limit_values.limits_limit_values.limit_values.*.availability_domain) != 0 ? data.oci_limits_limit_values.limits_limit_values.limit_values[0].availability_domain : data.oci_identity_availability_domains.availability_domains.availability_domains[0]["name"]
}
57 changes: 57 additions & 0 deletions workshops/dcms-db/config/terraform/core_instance.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Copyright © 2020, Oracle and/or its affiliates.
# All rights reserved. The Universal Permissive License (UPL), Version 1.0 as shown at http://oss.oracle.com/licenses/upl

// Get the latest Oracle Linux image
data "oci_core_images" "images" {
compartment_id = var.compartment_ocid
operating_system = var.compute_os
operating_system_version = var.linux_os_version
shape = local.compute_shape

filter {
name = "display_name"
values = ["^.*Oracle[^G]*$"]
regex = true
}
}

resource "oci_core_instance" "instance" {
compartment_id = var.compartment_ocid
display_name = format("%s-ords-core", var.proj_abrv)
availability_domain = local.availability_domain
shape = local.compute_shape
dynamic "shape_config" {
for_each = local.is_flexible_shape ? [1] : []
content {
baseline_ocpu_utilization = "BASELINE_1_2"
ocpus = var.compute_flex_shape_ocpus[var.size]
// Memory OCPU * 16GB
memory_in_gbs = var.compute_flex_shape_ocpus[var.size] * 16
}
}
source_details {
source_type = "image"
source_id = data.oci_core_images.images.images[0].id
}
agent_config {
are_all_plugins_disabled = false
is_management_disabled = false
is_monitoring_disabled = false
plugins_config {
desired_state = "ENABLED"
name = "Bastion"
}
}
// If this is ALF, we can't place in the private subnet as need access to the cloud agent/packages
create_vnic_details {
subnet_id = local.is_always_free ? oci_core_subnet.subnet_public.id: oci_core_subnet.subnet_private[0].id
assign_public_ip = local.is_always_free
nsg_ids = [oci_core_network_security_group.security_group_ssh.id, oci_core_network_security_group.security_group_ords.id]
}
metadata = {
ssh_authorized_keys = file(var.ssh_public_key_file)
}
lifecycle {
ignore_changes = all
}
}
146 changes: 146 additions & 0 deletions workshops/dcms-db/config/terraform/core_nsg.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
# Copyright © 2020, Oracle and/or its affiliates.
# All rights reserved. The Universal Permissive License (UPL), Version 1.0 as shown at http://oss.oracle.com/licenses/upl

#####################################################################
## Always Free + Paid Resources
#####################################################################
// Security Group for SSH
resource "oci_core_network_security_group" "security_group_ssh" {
compartment_id = var.compartment_ocid
vcn_id = oci_core_vcn.vcn.id
display_name = format("%s-security-group-ssh", var.proj_abrv)
}
// Security Group for SSH - EGRESS
resource "oci_core_network_security_group_security_rule" "security_group_ssh_egress" {
network_security_group_id = oci_core_network_security_group.security_group_ssh.id
direction = "EGRESS"
protocol = "6"
destination = "0.0.0.0/0"
destination_type = "CIDR_BLOCK"
}
// Security Group for SSH - INGRES
resource "oci_core_network_security_group_security_rule" "security_group_ssh_ingress_TCP22" {
network_security_group_id = oci_core_network_security_group.security_group_ssh.id
direction = "INGRESS"
protocol = "6"
source = "0.0.0.0/0"
source_type = "CIDR_BLOCK"
tcp_options {
destination_port_range {
max = 22
min = 22
}
}
}

// Security Group for Load Balancer (lb)
resource "oci_core_network_security_group" "security_group_lb" {
compartment_id = var.compartment_ocid
vcn_id = oci_core_vcn.vcn.id
display_name = format("%s-security-group-lb", var.proj_abrv)
}
// Security Group for Load Balancer (lb) - EGRESS
resource "oci_core_network_security_group_security_rule" "security_group_lb_egress" {
network_security_group_id = oci_core_network_security_group.security_group_lb.id
direction = "EGRESS"
protocol = "6"
destination = "0.0.0.0/0"
destination_type = "CIDR_BLOCK"
}
// Security Group for Load Balancer (lb) - INGRESS
resource "oci_core_network_security_group_security_rule" "security_group_lb_inress_TCP80" {
network_security_group_id = oci_core_network_security_group.security_group_lb.id
direction = "INGRESS"
protocol = "6"
source = "0.0.0.0/0"
source_type = "CIDR_BLOCK"
tcp_options {
destination_port_range {
max = 80
min = 80
}
}
}
resource "oci_core_network_security_group_security_rule" "security_group_lb_inress_TCP443" {
network_security_group_id = oci_core_network_security_group.security_group_lb.id
direction = "INGRESS"
protocol = "6"
source = "0.0.0.0/0"
source_type = "CIDR_BLOCK"
tcp_options {
destination_port_range {
max = 443
min = 443
}
}
}

// Security Group for ORDS
resource "oci_core_network_security_group" "security_group_ords" {
compartment_id = var.compartment_ocid
vcn_id = oci_core_vcn.vcn.id
display_name = format("%s-security-group-ords", var.proj_abrv)
}
// Security Group for ORDS - EGRESS
resource "oci_core_network_security_group_security_rule" "security_group_ords_egress_grp" {
network_security_group_id = oci_core_network_security_group.security_group_ords.id
direction = "EGRESS"
protocol = "6"
destination = oci_core_network_security_group.security_group_ords.id
destination_type = "NETWORK_SECURITY_GROUP"
}
resource "oci_core_network_security_group_security_rule" "security_group_ords_egress" {
network_security_group_id = oci_core_network_security_group.security_group_ords.id
direction = "EGRESS"
protocol = "6"
destination = "0.0.0.0/0"
destination_type = "CIDR_BLOCK"
}
// Security Group for ORDS - INGRESS
resource "oci_core_network_security_group_security_rule" "security_group_ords_ingress_TCP8080" {
network_security_group_id = oci_core_network_security_group.security_group_ords.id
direction = "INGRESS"
protocol = "6"
source = var.public_subnet_cidr
source_type = "CIDR_BLOCK"
tcp_options {
destination_port_range {
max = 8080
min = 8080
}
}
}

#####################################################################
## Paid Resources
#####################################################################
resource "oci_core_network_security_group" "security_group_adb" {
count = local.is_always_free ? 0 : 1
compartment_id = var.compartment_ocid
vcn_id = oci_core_vcn.vcn.id
display_name = format("%s-security-group-adb", var.proj_abrv)
}
// Security Group for ADB - EGRESS
resource "oci_core_network_security_group_security_rule" "security_group_adb_egress" {
count = local.is_always_free ? 0 : 1
network_security_group_id = oci_core_network_security_group.security_group_adb[0].id
direction = "EGRESS"
protocol = "6"
destination = var.private_subnet_cidr
destination_type = "CIDR_BLOCK"
}
// Security Group for ADB - INGRESS
resource "oci_core_network_security_group_security_rule" "security_group_adb_ingress_TCP1522" {
count = local.is_always_free ? 0 : 1
network_security_group_id = oci_core_network_security_group.security_group_adb[0].id
direction = "INGRESS"
protocol = "6"
source = var.private_subnet_cidr
source_type = "CIDR_BLOCK"
tcp_options {
destination_port_range {
max = 1522
min = 1522
}
}
}
Loading