rr is unsafe to run on potentially malicious code

[rocallahan]

From @DemiMarie:

Is bare metal reasonably usable for browsers? rr allows sandbox escapes, so I don’t know if it is reasonable to use it for a browser that is going to be accessing untrusted web content, which is the usual case.

@rocallahan I can say that in my experience (developer of https://github.com/QubesOS), there are times when I wanted to use rr, but I was never able to do so because of the performance counter requirement. Xen doesn’t expose performance counters in VMs, and it is a type 1 hypervisor so everything is a VM. Furthermore, rr allows sandbox escapes, so using it for web browsers accessing untrusted web content is ill-advised outside of a test system.

[rocallahan]

rr allows sandbox escapes, so I don’t know if it is reasonable to use it for a browser that is going to be accessing untrusted web content, which is the usual case.

Actually the usual case for Firefox and Chrome devs using rr is that they're debugging test cases which are known to be non-hostile, so that's fine.

But recording a browser session involving potentially hostile Web content seems fine too. rr does not flat-out disable the browser sandbox, it just pokes a very specific small hole in it. The odds that a real Web site is carrying a malicious payload that specifically targets that hole poked by rr must be infinitesimal. Why would anyone do that? I'm sure you are much more likely to encounter Web content that uses a regular bug-exploiting sandbox escape.

If you're debugging a browser using rr and you're worried about an attacker trying to specifically target you, you should run the browser and rr in a VM guest using something KVM-based. An AWS EC2 VM works fine. But against such a sophisticated attacker, you should probably do that even if you're not using rr.

Xen doesn’t expose performance counters in VMs,

Sure it does: https://github.com/mirage/xen/blob/master/xen/arch/x86/cpu/vpmu.c
However Xen's PMU virtualization (as of 2016) had a bug that prevented rr from working properly: https://lists.xen.org/archives/html/xen-devel/2016-10/msg01285.html
It probably could be fixed, and maybe has been fixed.

[DemiMarie]

rr allows sandbox escapes, so I don’t know if it is reasonable to use it for a browser that is going to be accessing untrusted web content, which is the usual case.

Actually the usual case for Firefox and Chrome devs using rr is that they're debugging test cases which are known to be non-hostile, so that's fine.

I was not aware of that. My use-case was tracking down with crashes found by end-users, such as QubesOS/qubes-issues#8960 (Firefox crashes for no apparent reason).

But recording a browser session involving potentially hostile Web content seems fine too. rr does not flat-out disable the browser sandbox, it just pokes a very specific small hole in it. The odds that a real Web site is carrying a malicious payload that specifically targets that hole poked by rr must be infinitesimal. Why would anyone do that? I'm sure you are much more likely to encounter Web content that uses a regular bug-exploiting sandbox escape.

Attackers benefit massively from automation. An attacker might be using a payload that tries a whole bunch of different exploits. Spyware like Pegasus does exactly that.

If you're debugging a browser using rr and you're worried about an attacker trying to specifically target you, you should run the browser and rr in a VM guest using something KVM-based. An AWS EC2 VM works fine.

Indeed it does, though only in the largest (and thus most expensive) sizes (because AWS EC2 blocks performance counter use on smaller VMs for security reasons).

But against such a sophisticated attacker, you should probably do that even if you're not using rr.

That is why I use and develop Qubes OS, which does the same thing but with Xen VMs.

Xen doesn’t expose performance counters in VMs,

Sure it does: https://github.com/mirage/xen/blob/master/xen/arch/x86/cpu/vpmu.c However Xen's PMU virtualization (as of 2016) had a bug that prevented rr from working properly: https://lists.xen.org/archives/html/xen-devel/2016-10/msg01285.html It probably could be fixed, and maybe has been fixed.

The functionality exists, but it is not supported security-wise, which means that I cannot ask end-users to use it. See XSA-163: virtual PMU is unsupported.

Is this just me wanting to use rr for something it was never meant for?

[rocallahan]

Attackers benefit massively from automation. An attacker might be using a payload that tries a whole bunch of different exploits. Spyware like Pegasus does exactly that.

Sure, but someone would have to develop an exploit specifically targeting rr first, i.e. to specifically target browser developers who use rr on public Web content, which is a VERY small set of people. I probably know most of their names! And the chance that you'll ever get those users to load your payload while they're recording the browser with rr is still incredibly small.

Indeed it does, though only in the largest (and thus most expensive) sizes

rr works in c5d.9xlarge instances which cost < $2/hour. That's essentially nothing compared to the value of a developer's time. I understand that that doesn't work for Qubes-specific issues that you want to debug, but it would work for just about everyone else.

The functionality exists, but it is not supported security-wise, which means that I cannot ask end-users to use it.

So you want end-users to send you rr recordings of the Web browser for debugging? There are bigger problems with that. In particular it's difficult to ensure that those recordings don't contain their private information.

Is this just me wanting to use rr for something it was never meant for?

I guess so? We do specifically warn against using rr to debug malicious code: https://github.com/rr-debugger/rr/wiki/Usage#malicious-code

It might be possible to harden rr for safe recording and replaying of malicious code, but it would be a lot of work to implement and more importantly to test, and it just doesn't seem worth it when most users can just run it in a relatively cheap AWS VM instead.

[DemiMarie]

The functionality exists, but it is not supported security-wise, which means that I cannot ask end-users to use it.

So you want end-users to send you rr recordings of the Web browser for debugging? There are bigger problems with that. In particular it's difficult to ensure that those recordings don't contain their private information.

I was thinking of a separate (fresh) profile with no information saved, but you are correct. I don’t think there is a good solution here, beyond what ABRT does.