followup: BS / prototype lies #61
Comments
|
actually, plugins and mimetypes are bypassable in FF since FF52+ only allowed flash, and now flash is ripped out: edit: actually only FF85+ (where the answer is always none), since I guess someone could fake Flash randomly |
|
@abrahamjuliot can you check my |
|
|
Oh. Is that all possum does? navigator stuff? I was worried about false positives in the (iframe) window properties. I guess I better install possum, but I could never get it to do anything (maybe it hates me) |
|
It returns gibberish on canvas Testing requires a single script to trigger the fingerprinting threshold and then the Possom targets that single script. [EDIT: as far as know. Cowlicks is the expert there] On TZP, it looks like the canvas test triggers the threshold, so the noise is only applied to that script js file. I used to see all of the noise on creep, but then once I began using an iframe window (for most metrics), I realized Privacy Possom does not protect iframes - all of it is bypassed there. One of the subtle advantages of separating scripts in small chunks (as on TZP) is site scrappers and extensions cannot easily bypass or block a script URL for fingerprinting. |
|
the other way FPing scripts can't be blocked as scripts, is when they're first party and part of an essential js file. FPJS2 encourages this
Can't get it to trigger. Maybe it takes a few goes over several browser sessions (I did try reruns, reloads to make it learn). Cleaning up the false positives is not super essential: the overall fingerprint is usually already changed - the backend analysis can just treat that metric as suspicious if it's an outlier and the suspect list is not empty |
|
check it out: b30b4fe - edit: read my comment at the end |
|
that was fun: b36f53b .. enjoy the read
We only need the bypass when someone is lying: if we don't get a bypass, then we fall back to prototype lies, so at least the stability of the section hash's zoom resistance is only two values (when someone lies). I have ideas to cover the non-bypassed results I tested my screen at various zoom levels
Anyway, I debugged and output on resize and got some missing ranges (as long as one is invalid I can't use the bypass)
I don't know exactly at what combination of system-scaling and zoom-level that the bypass fails (remember, it only matters if someone lies), but as you can see, as soon as you zoom down, the numbers scale up way too fast: so increasing the upper limit from I'm not sure what happens with smaller screens and/or system-scaling (usually people scale up due to eyesight etc). I think it improves, or maybe the valid-zoom range just shifts but doesn't grow Eager to here your thoughts. Also this bit "I have ideas to cover the non-bypassed results" Lines 1778 to 1780 in b36f53b So, if prototype lies shows some BS is going on, we start bisecting with css rules: NFI on the perf, or if we should first bisect by if it is under or over our range (500-2560) and then go from there |
|
@abrahamjuliot Hmmm, so Chameleon seems to remove navigator keys It's spoofing as Chrome, so it removes const get_navKeys = () => new Promise(resolve => {
// reset
navKeys = {}
// build
try {
let keys = Object.keys(Object.getOwnPropertyDescriptors(Navigator.prototype))
// true/fake keys
if (runSL) {keys.push("iamfake")} // simulate fake keys being added
let trueKeys = keys
let lastKeyIndex = keys.length
let fakeKeys = []
// orig minus constructor
let allKeys = keys
allKeys = allKeys.filter(x => !["constructor"].includes(x))
navKeys["allKeys"] = allKeys
if (isFF) {
// constructor is always last
lastKeyIndex = keys.indexOf("constructor")
trueKeys = keys.slice(0, lastKeyIndex+1)
fakeKeys = keys.slice(lastKeyIndex+1)
} else if (isEngine == "blink") {
// last key inconsistent
let knownPoison = ["SharedWorker","Worker","buildID","getVRDisplays","activeVRDisplays","oscpu","iamfake"]
trueKeys = keys.filter(x => !knownPoison.includes(x))
fakeKeys = keys.filter(x => knownPoison.includes(x))
}
// remove constructor
trueKeys = trueKeys.filter(x => !["constructor"].includes(x))
// set
navKeys["trueKeys"] = trueKeys
navKeys["fakeKeys"] = fakeKeys
// set brave
isBraveMode = "unknown"
if (check_navKey("brave")) {
isBrave = true
Promise.all([
get_isBraveMode(),
]).then(function(results){
isBraveMode = results[0]
return resolve()
})
} else {
return resolve()
}
} catch(e) {
console.error("get_navKeys", e.name, e.message)
return resolve()
}
})Can we harden navKeys? I thought extensions could only add, not remove? Yikes! I'm wondering if we could use a knownGood and add in missing items. e.g. in FF there's no pref or legit means to remove some of these items: such as those two Two ways
|
|
My initial thoughts on deleting properties were upside down. Technically, extensions can delete or rewrite any object and/or object key values. As far as I know, there is no way to obtain the original object apart from relying on an unprotected iframe context. Adding missing nav keys is possible if the known version is used to determine the expected keys and their order. At a minimum, we could restore long-standing/static keys as a partial bypass, but the proper order would have to be manually constructed. I only looked at Tor Browser and FF Nightly, but the order of long-standing keys is consistent and new keys in FF Nightly do not change the order of the long-standing keys Example: |
|
Thanks. I was going to check a few linux VMs and debug some output on android, but I too believe sorting is OK to use That said: it's important to not lose any entropy that the extension brings: hence we record the lies and fake keys etc: just not in the main FP (for stable data analysis). navKeys is special, because I use it elsewhere, but not for absolutely everything. It's good we can weed out those after constructor in FF, and knownPoison in blink .... adding in items we know exist and don't check for (just error handling etc like the useragent parts) is probably moot .. might be easier to just note missing items and tag it as untrustworthy. And after all that, if an extension did mutate it, so be it: at least we don't lose the entropy I wonder how else we can harden navKeys: iframe test, test each missing line item in try catches ... food for thought |
|
edit: 4d185f3 I don't think there is any legitimate way (outside of special builds) to remove some items we just add missing expected keys to the "fake" list: the engine already tells you if it was added or removed: e.g. Does expectedKeys look alright to you? if (isFF) {
// constructor is always last
// track added keys
lastKeyIndex = keys.indexOf("constructor")
trueKeys = keys.slice(0, lastKeyIndex+1)
fakeKeys = keys.slice(lastKeyIndex+1)
// track missing keys: onLine incl because this is served over HTTPS
let expectedKeys = ["appCodeName","appName","buildID","hardwareConcurrency",
"language","languages","mimeTypes","onLine","oscpu","platform","plugins",
"product","productSub","userAgent","vendor","vendorSub"]
let missingKeys = expectedKeys.filter(x => !keys.includes(x))
trueKeys = trueKeys.concat(missingKeys)
fakeKeys = fakeKeys.concat(missingKeys)
}
//snip
// set
navKeys["trueKeys"] = trueKeys.sort()
navKeys["fakeKeys"] = fakeKeys.sort()
navKeys["allKeys"] = allKeys.sort() |
|
The expected keys look good. If there is no pref removing these keys, perhaps we could add them: appVersion (deprecated but still standing) |
|
expected keys is FF only, but we could populate it twice, once for gecko, once for blink I need to test more, but I think you're right: all pass in TB alpha FWIW appVersion (deprecated but still standing) - not deprecated in FF, totally missed this |
|
@abrahamjuliot ... https://github.com/arkenfox/TZP/blob/master/js/prototypeLies.js#L509 could/should we add |
|
Yes, a script can modify the constants. We could add them to the prototype test, but it's quicker to do an equality check and then bypass the lie. |
|
Sorry, I'm confused: What do you mean an equality check. The whole point is that we don't know the right answer, so we need to check if the "function" is tampered with and the result deemed untrustworthy (regardless of if it ultimately tampered with or not) - e.g. hardwareConcurrency has no means to verify via other means (well, not timely ones) Or do you mean it's different because it's a constant and not a function. And to test is to check the result against known results? Since browsers will only be with 1 or 2 floating points of each other? Forget cydec with it's random per execution .. what if it did it persistent noise per eTLD+1 per tab session |
|
Correct, these are just numbers and not functions. In the case of a Math constant, we can check if it yields the correct constant value, and as far as I know, Math constants are consistent across JS engines and not implementation-dependent (contrary to Math functions). const mathE = 2.718281828459045
const liedMathE = Math.E !== mathE
if (liedMathE) {
// use mathE bypass
}const mathConstantsValid = (
Math.E == 2.718281828459045 &&
Math.LN2 == 0.6931471805599453 &&
Math.LN10 == 2.302585092994046 &&
Math.LOG2E == 1.4426950408889634 &&
Math.LOG10E == 0.4342944819032518 &&
Math.PI == 3.141592653589793 &&
Math.SQRT1_2 == 0.7071067811865476 &&
Math.SQRT2 == 1.4142135623730951
)
That would be harder to detect. There's also persistent noise cross-site per time seed and static profiles per domain. |
|
Thanks. That's what I thought. If cydec did a persistent cached noise (not random per execution) and covered e.g. cos(-1) and Math.E etc, well then I guess in a math test with a few items, at least something should end up in prototype lies (cydec lists 19 math items)
I hope so: at least they should be with 16 decimal places - which is what I was looking at in that other issue Math.E == 2.718281828459045 - I think i've seen this at 2.7182818284590455 (or maybe that was the polyfill talking) |
|
PS: just updated cydec: release notes says he opened a github tracker issue: https://github.com/heilig-defense/anti-fp/issues |
|
fyi - It appears, both CB and Chameleon use Proxies now. I have a patch that detects Proxies in Firefox. It's very basic, so I'm working on making it more robust (if possible). const getIncompatibleProxyTypeErrorLie = apiFunction => {
const isFirefox = 3.141592653589793 ** -100 == 1.9275814160560185e-50 // this could be improved
try {
apiFunction.arguments
apiFunction.caller
return true
} catch (error) {
return (
error.constructor.name != 'TypeError' ||
(isFirefox && /incompatible\sProxy/.test(error.message)) ? true : false
)
}
}
getIncompatibleProxyTypeErrorLie(HTMLCanvasElement.prototype.toDataURL) |
|
Sounds exciting :) Does this have any bearing on tests, prototype lies etc? I'm a little lost on the implications here. Or is this yet one more piece of entropy that we can add and yet another pitfall of extensions? |
|
Yes, it's another extension pitfall, but only has bearing on the prototype tests. As of a recent CB update to Proxies, the prototype tampering is near undetected (only Side note [edit after reading: I'm terrible at making sense of this one]: I'm also looking at providing a secondary reduced detail list that filters out proxy detection on
|
|
Nice 👍 I love how CB works with So I should use Is that right? edit: Ahhh, OK. I just tested the old version, and I see what you mean, with CB only showing 1 lie (toBlob). So I guess the logic here is
I think I'll add a global tampering list to the fingerprint section, not sure what to call it .. This is why I ultimately prefer to use other methods: known results, multiple ways to measure, two-pass, the patented dom-shift-of-doom, feature detection, game theory + logic (e.g. touch support is false on android, WTF?) and I see you replied below... |
|
That's right. |
|
I just edited my comment above. Not sure what you mean by "directly tampered with" vs "indirect tampering" (edit: they're both "tampering", i.e untrustworthy: not everyone will use a proxy like CB) - my understanding is if it's in the lieList it can't be trusted |
|
I use |
|
OK, still confusing :) But yeah ... lieList "may trust some" has always been the issue (there's some logic to be had here, but IDK if it's worth it)
Anyway, I'm about to add the Good work my padawan grasshopper .. have some 🍰 |
|
After known lies (math, dom-shift-of-doom etc) I'll probably include a global TBH, there doesn't seem to be much that can't be detected as a known lie: textmetrics I have solutions for, audio we seem to know it's a limited set of results, fonts have numerous measuring methods, domrect and canvas are covered, feature detection gets almost all useragent (firefox), and css pseudo gets screen etc Lines 37 to 39 in 2ead93f |
|
Cydec is 177 lies in both .. aren't we meant to be sorting (I could have sworn we added this to prototype)? the hashes differ. I mean, I can easily sort mine, but it's better if we bake it into the return since we use the same "library"
edit Line 680 in 2ead93f return {
lieList: Object.keys(props).sort(),
lieDetail: props,
lieCount: Object.keys(props).reduce((acc, key) => acc + props[key].length, 0),
propsSearched,
// filter out lies on Function.prototype.toString
tamperingList: (props => {
// here, add sort()
return Object.keys(props).filter(key => {
const totalTamperingLies = getCountOfNonFunctionToStringLies(props[key])
if (!totalTamperingLies) {
return false
}
return true
})
})(props)
}edit: @abrahamjuliot ^^ in case you missed it |
|
And looks like Chameleon is 55 in both (in my chameleon config) .. so I'm starting to think proxyLies is always the way to go, which is kinda what you said, I need to drink on it |
NoScript throws 2 prototype lies. I'm not sure how unique they are against similar extensions, but we can hash the detail pattern and estimate if NoScript is on in TB. |
|
Yeah, I saw TB returning two lies (wasn't sure when it started, and didn't realize it was NS) .. I just ignored it, because the lies weren't anything I use: currently I'm only using some navigator lies (hardware concurrency and a bunch of user agent stuff), I think that's all not sure if estimating if NS is on in TB is worth it? Do you mean if the extension has been enabled/disabled? I don;t think anyone would bother to disable. Or do you mean if they whitelisted/blocked a page, in which case, I think the lies would still persist (?) and it wouldn't be a stable metric ... call me confused, what did you mean Oooohhhh .. and while I have you here, do you have a Mac, I desperately need a font test - I've just split the mac list of 755 fonts up
If you're not sure what these entail, it's to reduce the OS font list in TZP to the bare minimum: either the |
This estimation or unmasking is non-essential. In the case of NS, the lies persist in both whitelist/block settings.
I do. It's my trusty Mac VMware running in Windows on a MacBook Pro device (I deleted the real OS 🙃). I'll jump on this in #34. |
|
I think Cydec decided to not lie about Firefox any more. Latest update always seems to return FF90 (and thus a bunch of things aren't spoofed as a result) .. I think from memory I had 21 lies detected, 19 bypassed (hadn't finished TBH) .. now it's 5 lies, 4 bypassed added to my todo list
|
|
@abrahamjuliot FYI
some quick notes, haven't checked much more
One thing I noticed was that ac1c639 - pretty sure you use |
|
catching performance.now tampering is intermittent. We used this for cydec, it's patchy for JS Redirector if (Math.trunc(performance.now() - performance.now()) !== 0) {
// set vars, record stuff etc, dodgy MFers
if (gRun) {gMethods.push("_global:performance.now:tampered")}
} |
OK, I tweaked that test and since it's not part of the global fingerprint, then I won't bother with any lies or bypasses. Instead I will just use some logic and add a test
logic
JShelter on level 3 rounds to hundreds, and on lower level(s?) looks like 10ms, IDK but I do catch it out, but don't record anything since it's not part of the global FP Not sure what to do about hardening isPerf is which is part of get_isRFP(). isPerf is part of the global FP when it catches BShittery. isPerf = true // assume no fuckery
if (Math.trunc(performance.now() - performance.now()) !== 0) {
isPerf = false
if (gRun) {gMethods.push("_global:performance.now:tampered")}
}I also used isPerf to not output performance stats if false, but that was because cydec was so far off (thousands of ms) resetting global vars on every run (section/global) because they can change: so mindful not to add too much timing wasting setTimeout(function() {
gt0 = performance.now()
Promise.all([
get_isRFP(),
get_navKeys(),
outputPrototypeLies(),
get_isOS64(),
]).then(function(results){
output()
})
}, delay)time for a ☕ |
|
apparently cydec had a bug: it's back to |
|
@abrahamjuliot do you know of any chrome extensions that fake keyboard? I want to test my keyboard BS detection :) - edit cdbfbee |
|
I'm not aware of any. I have not yetexplored keyboard focused extensions. There might be something out there. |
|
|
@abrahamjuliot can we add this? to detect inner/outer window BS ? |
|
This should work, with the capital searchLies(() => Window, {
target: [
'innerWidth',
'innerHeight',
'outerWidth',
'outerHeight',
]
}) |
|
doesn't add anything, at least with Cydec |
|
FWIW, cydec alters HTMLElementKeys my solution is going to be
not sure if any entropy will be lost by sorting |
|
just a cydec update, and I am by no means finished here with unTruths and ReTruths I honestly believe that trying to using the web in cydec total mode must the world's shittiest experience with site breakage basically set to max
|
|
@abrahamjuliot so I was checking engine props in edgeHTML (windows 10 VM)
Anyway, I fired up TZP itself out of interest and .. well, ugh. Returns 148 lies, and prereq takes like 20+ seconds
after the patch, overall perf (the entire TZP) is more like 2 to 3 seconds, and I do get six false positive lies (domrect x 4, keyboard, plugins) which is not your problem, and I can code/fix those |
|
FYI: here are edgeHTML unique properties AFAICT. I get that prototypeLies calculates the engine on the fly (TZP has already set isEngine). I've also listed that it has a mix of blink-only, gecko-only, and webkit-only properties (and pairs of onlys is a total mess as well). IDK if prototypeLies is meant to support edgeHTML, but my strategy here would be to test for edgeHTML only first, then blink/gecko/webkit Line 389 in 5b37439 |
new thread, the last one #35 is getting too long and noisy
No need to list items: it's a work in progress as I make each section more bulletproof to errors, AOPR, etc. The logic is to either expose the real value via other methods/equivalency and/or detect that it's a lie: first through methods like known values etc, and lastly as untrustworthy via prototype lies
cydec total mode: 17
known lies(and not even close to being fully realized)I also decided to keep track of bypasses
xmeans I've coded recording the bypass [edit: updated]starting to like this game.. so many lies, so little time .. so many ideas
The text was updated successfully, but these errors were encountered: