Skip to content

Filter PII rather than removing headers entirely - #1161

Merged
ste93cry merged 17 commits into
getsentry:developfrom
soundsgoodsofar:soundsgoodsofar-pii-redaction
Feb 1, 2021
Merged

ste93cry merged 17 commits into
getsentry:developfrom
soundsgoodsofar:soundsgoodsofar-pii-redaction

Conversation

@soundsgoodsofar

Copy link
Copy Markdown
Contributor

It's often very useful to know what headers were passed, even without the value. Currently the PII filter is removing the key and value entirely. There shouldn't really be any security concern with passing the redacted header keys. I'm currently hardcoding '[Filtered]' so that the display matches in the Sentry UI. Ideally I would pass a null value, but that currently shows up as "" in Sentry.

An even better solution would be to provide a mechanism to inject a closure for applying PII redaction, so that people could do what they want if they don't like the default. There are some problems in how the sentry-laravel package does its Service Provider configuration that make swapping those out much more difficult than need be (I'll open an issue on that project as well).

I want to also add that the Integration classes have some pretty bad design patterns. Combined with the extensive use of final and private keywords, it makes fixes like this way more difficult than they need to be.

Example:
$keysToRemove = ['authorization', 'cookie', 'set-cookie', 'remote_addr'];

Things like that should be determined by config, not hard-coded into functions. Combined with private instead of protected, it makes changes far more complicated than they should be.

@ste93cry

ste93cry commented Jan 6, 2021

Copy link
Copy Markdown
Contributor

There shouldn't really be any security concern with passing the redacted header keys

I agree 👍

An even better solution would be to provide a mechanism to inject a closure for applying PII redaction, so that people could do what they want if they don't like the default.

You can use the before_send option or a custom event processor to make changes to the event while enabling the send_default_pii option: in this way, you have full control over what to scrub and what not

Things like that should be determined by config, not hard-coded into functions.

In the SDK 1.x the list of keys to redact was configurable, however since then it has been decided that SDKs should not be responsible for data scrubbing besides a really minimal config and the work has instead been relegated to Relay. Since this is a Unified API-wide decision, there are no plans to go another way for this specific SDK

@soundsgoodsofar

Copy link
Copy Markdown
Contributor Author

You can use the before_send option or a custom event processor to make changes to the event while enabling the send_default_pii option: in this way, you have full control over what to scrub and what not

Things like that should be determined by config, not hard-coded into functions.

In the SDK 1.x the list of keys to redact was configurable, however since then it has been decided that SDKs should not be responsible for data scrubbing besides a really minimal config and the work has instead been relegated to Relay. Since this is a Unified API-wide decision, there are no plans to go another way for this specific SDK

Makes sense, but I would still argue that this line in RequestIntegration should pull from config. If you don't think redacting PII is your library's job, that's understandable. But if you are going to offer a solution, I don't personally see any reason it shouldn't be adjustable in config.

$keysToRemove = ['authorization', 'cookie', 'set-cookie', 'remote_addr'];

Anyway, for the scope of this PR, is the change to redact the headers acceptable? Should it be null vs '[Filtered]'? If there's appetite for this I'll fix the unit tests (they're just failing because they expect the old values).

@ste93cry

ste93cry commented Jan 6, 2021

Copy link
Copy Markdown
Contributor

I don't personally see any reason it shouldn't be adjustable in config

The reason is that there is already a tool more suitable and tailored for this kind of thing, which is Relay. Making this configurable besides a simple boolean will just mean that at some point another person will came and ask for more configuration, different configuration or whatever else and we don't want to open this Pandora's box.

is the change to redact the headers acceptable?

As I said before, I agree so that's a yes 😃

@soundsgoodsofar

Copy link
Copy Markdown
Contributor Author

Updated the unit tests to account for the changes.

Up to you guys, but I would suggest that if the guidance is to use Relay and there is not an intention to build out support for the existing 'send-default-pii' pattern, that pattern should be marked as deprecated, and that information should be added to the documentation about using send-default-pii.

Thanks for your replies!

@ste93cry ste93cry left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you 🙏 both add a CHANGELOG entry and target this PR towards the develop branch?

Comment thread src/Integration/RequestIntegration.php Outdated
Comment thread src/Integration/RequestIntegration.php Outdated
Comment thread src/Integration/RequestIntegration.php Outdated
@ste93cry ste93cry added this to the 3.2 milestone Jan 8, 2021
@soundsgoodsofar
soundsgoodsofar force-pushed the soundsgoodsofar-pii-redaction branch 3 times, most recently from a7616e0 to fd1908b Compare January 8, 2021 20:25
@soundsgoodsofar
soundsgoodsofar changed the base branch from master to develop January 8, 2021 20:31
@soundsgoodsofar

Copy link
Copy Markdown
Contributor Author

@ste93cry okay I've rebased onto develop and made changes. I went ahead and moved those headers into options--let me know if you don't like that and I can pull that back out.

Code coverage is failing slightly but I'm not sure what else I can do there.

@ste93cry

Copy link
Copy Markdown
Contributor

@soundsgoodsofar do you still have plans to finish the work? Otherwise, if it's ok for you I would like to do it for you so that we can merge the PR

@ste93cry
ste93cry force-pushed the soundsgoodsofar-pii-redaction branch from aaa9a36 to a2ab1da Compare January 29, 2021 19:09
@ste93cry
ste93cry force-pushed the soundsgoodsofar-pii-redaction branch from a2ab1da to 8d4570a Compare January 29, 2021 19:28
@ste93cry
ste93cry requested a review from Jean85 January 29, 2021 20:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants