expand data handling (#663)

Author: bruno-garcia

src/docs/sdk/basics.mdx

@@ -1,6 +1,6 @@
 ---
 title: Development Basics
-sidebar_order: 0
+sidebar_order: 2
 ---
 
 So you want to develop an SDK?  Before you get started here are some basics that are

src/docs/sdk/data-handling.mdx

@@ -1,23 +1,55 @@
 ---
 title: "Data Handling"
-sidebar_order: 5
+sidebar_order: 3
 ---
 
 Data handling is the standardized context in how we want SDKs help users filter data.
 
 ## Sensitive Data
 
-In older SDKs you might sometimes see elaborate constructs to allow the user to strip away sensitive data. Newer SDKs no longer have this feature as it turned out to be too hard to maintain per-SDK. Instead, only two simple config options are left:
+SDKs should not include PII or other sensitive data in the payload by default.
+When building an SDK we can come across to some API that can give useful information to debug a problem.
+In the event that API returns data considered PII, we guard that behind a flag called _Send Default PII_.
+This is an option in the SDK called [_send-default-pii_](https://docs.sentry.io/platforms/python/configuration/options/#send-default-pii)
+and is **disabled by default**. That means that data that is naturally sensitive is not sent by default.
 
-- [_send-default-pii_](https://docs.sentry.io/error-reporting/configuration/#send-default-pii) is **disabled by default**, meaning that data that is naturally sensitive is not sent by default. That means, for example:
+Some examples of data guarded by this flag:
 
-  - When attaching HTTP requests to events, "raw" bodies (bodies which cannot be parsed as JSON or formdata) are removed, and known sensitive headers such as `Authorization` or `Cookies` are removed too.
+  - When attaching HTTP requests to events
+    - Request Body: "raw" bodies (bodies which cannot be parsed as JSON or formdata) are removed
+    - HTTP Headers: known sensitive headers such as `Authorization` or `Cookies` are removed too.
+    - *Note* that if a user explicitly sets a request on the scope, nothing is stripped from that request. The above rules only apply to integrations that come with the SDK.
+  - User-specific information (e.g. the current user ID according to the used web-framework) is not sent at all.
+  - On desktop applications
+    - The username logged in the device is not included. This is often a person's name.
+    - The machine name is not included, for example `Bruno's laptop`
+  - SDKs don't set `{{auto}}` as `user.ip`. This instructs the server to keep the connection's IP address.*
 
-  - User-specific information (e.g. the current user ID according to the used webframework) is not sent at all.
+* Specifically about IP address, it's important to note that it's standard to log IP address of incoming connecting in services on the Internet.
+This not only allows security tools and operations to understand abuse coming from a single IP, like spam bots and other issues.
+But also developers to understand if issues in their application are being triggered by a single malicious source.
 
-  - Note that if a user explicitly sets a request on the scope, nothing is stripped from that request. The above rules only apply to integrations that come with the SDK.
+Sentry server is always aware of the connecting IP address and can use it for logging in some platforms. Namely JavaScript and iOS/macOS/tvOS.
+All other platforms require the event to include `user.ip={{auto}}` which happens if `sendDefaultPii` is set to true.
 
-- [_before-send_](https://docs.sentry.io/error-reporting/configuration/#before-send) can be used to register a callback with custom logic to remove sensitive data.
+Before sending events to Sentry, the SDKs should invokes callbacks. That allows users to remove any sensitive data client-side.
+
+- [`before-send` and `event-processors`](https://develop.sentry.dev/sdk/unified-api/#static-api) can be used to register a callback with custom logic to remove sensitive data.
+
+### Application State
+
+App state can be critical to help developers reproduce bugs. For that reason, SDKs often collect app state and append to events through auto instrumentation.
+
+When attaching data that could potentially include sensitive data or PII, it's important to:
+
+- Add a note on the docs to notify developers.
+- [Mark that part of the protocol on Relay](https://github.com/getsentry/relay/pull/1383) as such. This allows [data scrubbing](https://docs.sentry.io/product/relay/) to run on those fields.
+
+Some examples of auto instrumentation that could attach sensitive data:
+
+- A SQL integration that includes the query. If a user doesn't use parameterized queries, and appends sensitive data to it, the SDK could include that in the event payload.
+- Desktop apps including window title.
+- A Web framework routing instrumentation attaching route `to` and `from`.
 
 ## Variable Size
 

src/docs/sdk/features.mdx

@@ -1,6 +1,6 @@
 ---
 title: 'Expected Features'
-sidebar_order: 2
+sidebar_order: 4
 ---
 
 The following is a description of features that are commonly expected in Sentry SDKs.  Make sure to also
@@ -27,7 +27,7 @@ What scope means depends on the application, for a web framework it is most like
 
 ## Automatic Context Data
 
-Automatic addition of useful attributes such as `tags` or `extra` or specific `contexts`. Typically means the SDK hooks into a framework so that it can set attributes that are known to be useful for most users.
+Automatic addition of useful attributes such as `tags` or `extra` or specific `contexts`. Typically means the SDK hooks into a framework so that it can set attributes that are known to be useful for most users. Please check [Data Handling](data-handling) for considerations.
 
 ## Breadcrumbs
 

src/docs/sdk/index.mdx

@@ -6,13 +6,13 @@ The following is a guide for implementing a new Sentry SDK based on the new <Lin
 It covers the protocol for event submission as well as guidelines for how clients should
 typically look and behave.
 
+- <Link to="/sdk/overview/">Overview</Link>
 - <Link to="/sdk/philosophy/">Philosophy and Design Principles</Link>
 - <Link to="/sdk/basics/">Development Basics</Link>
-- <Link to="/sdk/overview/">Overview</Link>
+- <Link to="/sdk/data-handling/">Data Handling</Link>
 - <Link to="/sdk/unified-api/">Unified API</Link>
 - <Link to="/sdk/features/">Expected Features</Link>
 - <Link to="/sdk/event-payloads/">Event Payloads</Link>
-- <Link to="/sdk/data-handling/">Data Handling</Link>
 - <Link to="/sdk/store/">Store Endpoint</Link>
 - <Link to="/sdk/envelopes/">Envelopes</Link>
 - <Link to="/sdk/sessions/">Sessions</Link>

src/docs/sdk/overview.mdx

@@ -1,6 +1,6 @@
 ---
 title: Overview
-sidebar_order: 0
+sidebar_order: 1
 ---
 
 The following is a guide for implementing a new Sentry SDK. It covers the protocol for event submission as well as guidelines for how clients should typically look and behave.

src/docs/sdk/philosophy.mdx

@@ -46,3 +46,10 @@ At no point should a customer upgrade the library and find themselves in a situa
 ## Enable Customers
 
 While we generally should try to keep the API surfaces of SDKs reasonable small. At the same time we also need to make sure that we enable customers to achieve their goals. Think about cases that the SDK maybe wouldn't be able to solve out of the box. If there are enough APIs that customers can use our SDKs in more creative ways, we generally see this as a added benefit.
+
+## Handle PII and Sensitive Data with Care
+
+Some types of errors cannot be resolved without the data that was given to the program. 
+But make sure that auto instrumentations doesn't attach PII without an explicit opt-in from the user.
+The server must be aware of parts of the protocol that include PII to scrub them by default.
+Please check [Data Handling](data-handling) for more detail.

src/docs/sdk/rate-limiting.mdx

@@ -1,6 +1,6 @@
 ---
 title: 'Rate Limiting'
-sidebar_order: 3
+sidebar_order: 5
 ---
 
 Rate limits are communicated to SDKs via status codes and response headers. For regular rate limit responses, we emit a [`429`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/429) status code and specify a [`Retry-After`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After) header.

src/docs/sdk/signal-handlers.mdx

@@ -1,9 +1,11 @@
 ---
 title: Signal Handlers
+sidebar_order: 30
 ---
 
-Some Sentry SDKs have to work with signal handlers to trap processes before they
-crash.  This applies to mobile SDKs as well as the native SDK.  As signal handlers
+Native Sentry SDKs like sentry-native and sentry-cocoa have to work with signal handlers
+to trap processes before they crash.
+This applies to mobile SDKs as well as the native SDK.  As signal handlers
 are notoriously difficult to work with and the restrictions placed on us are
 quite limiting we have to partially bend the rules of what is acceptable.
 

src/docs/sdk/store.mdx

@@ -1,6 +1,6 @@
 ---
 title: Store Endpoint
-sidebar_order: 6
+sidebar_order: 50
 ---
 
 The store endpoint is used to send JSON event payloads to Sentry. It is located