Skip to content

Je/connect proxy support #15495

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Feb 5, 2025
Merged

Je/connect proxy support #15495

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
e560380
+ proxy request
jsendo Jan 14, 2025
37320f8
makeProxyRequest
jsendo Jan 15, 2025
5e45ac5
remove header and body requirements
jsendo Jan 28, 2025
8b8a8b4
Merge branch 'master' into je/connect-proxy-support
jsendo Feb 3, 2025
3ec43b5
clean up
jsendo Feb 4, 2025
1e87d88
Merge branch 'master' into je/connect-proxy-support
jsendo Feb 4, 2025
f73fe79
fix linting
jsendo Feb 4, 2025
9ab1113
change the function header to be more fetch-like
jsendo Feb 5, 2025
76a8ff7
PR feedback
jsendo Feb 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions packages/sdk/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,12 @@

# Changelog

## [1.3.3] - 2025-02-5

### Changed

- Add makeProxyRequest function to BaseClient

Comment on lines +5 to +10
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Enhance the changelog entry with more details.

The changelog entry should provide more context about the proxy request feature:

  1. Describe the purpose and benefits
  2. Document the new types added
  3. Include usage examples or migration guidance
 ## [1.3.3] - 2025-02-5
 
 ### Changed
 
-- Add makeProxyRequest function to BaseClient
+### Added
+
+- Added proxy request support to enable secure API calls through Pipedream's infrastructure
+  - New `makeProxyRequest` function in BaseClient for making proxy requests
+  - New types: `ProxyApiOpts`, `ProxyTargetApiOpts`, and `ProxyTargetApiRequest`
+
+Example usage:
+```typescript
+const response = await client.makeProxyRequest(
+  { searchParams: { external_user_id: "user123", account_id: "acc456" } },
+  {
+    url: "https://api.example.com/data",
+    options: {
+      method: "GET",
+      headers: { "Accept": "application/json" }
+    }
+  }
+);
+```

## [1.3.2] - 2025-02-3

### Changed
Expand Down
2 changes: 1 addition & 1 deletion packages/sdk/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "@pipedream/sdk",
"type": "module",
"version": "1.3.2",
"version": "1.3.3",
"description": "Pipedream SDK",
"main": "./dist/server.js",
"module": "./dist/server.js",
Expand Down
74 changes: 73 additions & 1 deletion packages/sdk/src/server/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing version update and CHANGELOG

import * as oauth from "oauth4webapi";
import {
Account, BaseClient, type AppInfo, type ConnectTokenResponse,
Account, BaseClient, type AppInfo, type ConnectTokenResponse, type RequestOptions,
} from "../shared/index.js";
export * from "../shared/index.js";

Expand Down Expand Up @@ -107,6 +107,48 @@ export type GetAccountByIdOpts = {
include_credentials?: boolean;
};

/**
* Options used to determine the external user and account to be used in Connect Proxy API
*/
export type ProxyApiOpts = {
/**
* Search parameters to be added to the proxy request. external_user_id and account_id are required.
*/
searchParams: Record<string, string>;
};
Comment on lines +113 to +118
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Enforce required fields in ProxyApiOpts type.

The type definition should enforce the required fields mentioned in the JSDoc comment.

 export type ProxyApiOpts  = {
   /**
    * Search parameters to be added to the proxy request.  external_user_id and account_id are required.
    */
-  searchParams: Record<string, string>;
+  searchParams: {
+    external_user_id: string;
+    account_id: string;
+    [key: string]: string;
+  };
 };
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
export type ProxyApiOpts = {
/**
* Search parameters to be added to the proxy request. external_user_id and account_id are required.
*/
searchParams: Record<string, string>;
};
export type ProxyApiOpts = {
/**
* Search parameters to be added to the proxy request. external_user_id and account_id are required.
*/
searchParams: {
external_user_id: string;
account_id: string;
[key: string]: string;
};
};


/**
* fetch-like options for the Target of the Connect Proxy Api Request
*/
export type ProxyTargetApiOpts = {
/**
* http method for the request
*/
method: "GET" | "POST" | "PUT" | "DELETE" | "PATCH";
/**
* http headers for the request
*/
headers?: Record<string, string>;
/**
* http body for the request
*/
body?: string;
};

/**
* object that contains the url and options for the target of the Connect Proxy Api Request
*/
export type ProxyTargetApiRequest = {
/**
* URL for the target of the request. Search parameters must be included here.
*/
url: string;
/**
* fetch-like options for the target of the Connect Proxy Request
*/
options: ProxyTargetApiOpts;
};

/**
* Creates a new instance of BackendClient with the provided options.
*
Expand Down Expand Up @@ -374,4 +416,34 @@ export class BackendClient extends BaseClient {
method: "GET",
});
}

/**
* Makes a proxy request to the target app API with the specified query parameters and options.
*
* @returns A promise resolving to the response from the downstream service
*/
public makeProxyRequest(proxyOptions: ProxyApiOpts, targetRequest: ProxyTargetApiRequest): Promise<string> {
const url64 = btoa(targetRequest.url).replace(/\+/g, "-")
.replace(/\//g, "_")
.replace(/=+$/, "");

const headers = targetRequest.options.headers || {};

const newHeaders = Object.keys(headers).reduce<{ [key: string]: string }>((acc, key) => {
acc[`x-pd-proxy-${key}`] = headers[key];
return acc;
}, {});

const newOpts: RequestOptions = {
method: targetRequest.options.method,
headers: newHeaders,
params: proxyOptions.searchParams,
}

if (targetRequest.options.body) {
newOpts.body = targetRequest.options.body
}

return this.makeConnectRequest(`/proxy/${url64}`, newOpts);
}
Comment on lines +425 to +448
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Improve error handling and input validation in proxy request implementation.

The implementation needs:

  1. URL validation
  2. Error handling for base64 encoding
  3. Validation for required search parameters
  4. Proper indentation
   public makeProxyRequest(proxyOptions: ProxyApiOpts, targetRequest: ProxyTargetApiRequest): Promise<string> {
+    // Validate required search parameters
+    if (!proxyOptions.searchParams.external_user_id || !proxyOptions.searchParams.account_id) {
+      throw new Error("external_user_id and account_id are required in searchParams");
+    }
+
+    // Validate URL
+    if (!targetRequest.url?.trim()) {
+      throw new Error("URL is required");
+    }
+
+    let url64: string;
+    try {
       url64 = btoa(targetRequest.url).replace(/\+/g, "-")
-      .replace(/\//g, "_")
-      .replace(/=+$/, "");
+        .replace(/\//g, "_")
+        .replace(/=+$/, "");
+    } catch (error) {
+      throw new Error(`Failed to encode URL: ${error.message}`);
+    }

     const headers = targetRequest.options.headers || {};
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
public makeProxyRequest(proxyOptions: ProxyApiOpts, targetRequest: ProxyTargetApiRequest): Promise<string> {
const url64 = btoa(targetRequest.url).replace(/\+/g, "-")
.replace(/\//g, "_")
.replace(/=+$/, "");
const headers = targetRequest.options.headers || {};
const newHeaders = Object.keys(headers).reduce<{ [key: string]: string }>((acc, key) => {
acc[`x-pd-proxy-${key}`] = headers[key];
return acc;
}, {});
const newOpts: RequestOptions = {
method: targetRequest.options.method,
headers: newHeaders,
params: proxyOptions.searchParams,
}
if (targetRequest.options.body) {
newOpts.body = targetRequest.options.body
}
return this.makeConnectRequest(`/proxy/${url64}`, newOpts);
}
public makeProxyRequest(proxyOptions: ProxyApiOpts, targetRequest: ProxyTargetApiRequest): Promise<string> {
// Validate required search parameters
if (!proxyOptions.searchParams.external_user_id || !proxyOptions.searchParams.account_id) {
throw new Error("external_user_id and account_id are required in searchParams");
}
// Validate URL
if (!targetRequest.url?.trim()) {
throw new Error("URL is required");
}
let url64: string;
try {
url64 = btoa(targetRequest.url).replace(/\+/g, "-")
.replace(/\//g, "_")
.replace(/=+$/, "");
} catch (error) {
throw new Error(`Failed to encode URL: ${error.message}`);
}
const headers = targetRequest.options.headers || {};
const newHeaders = Object.keys(headers).reduce<{ [key: string]: string }>((acc, key) => {
acc[`x-pd-proxy-${key}`] = headers[key];
return acc;
}, {});
const newOpts: RequestOptions = {
method: targetRequest.options.method,
headers: newHeaders,
params: proxyOptions.searchParams,
}
if (targetRequest.options.body) {
newOpts.body = targetRequest.options.body;
}
return this.makeConnectRequest(`/proxy/${url64}`, newOpts);
}

}
2 changes: 1 addition & 1 deletion packages/sdk/src/shared/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -995,8 +995,8 @@ export abstract class BaseClient {
};

return this.makeRequest(path, {
headers,
...opts,
headers,
});
}

Expand Down
Loading