Skip to content

[BUG] Some Android 14 QPR2 APKs not supported #3428

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
evowizz opened this issue Nov 16, 2023 · 12 comments · Fixed by #3452
Closed

[BUG] Some Android 14 QPR2 APKs not supported #3428

evowizz opened this issue Nov 16, 2023 · 12 comments · Fixed by #3452
Assignees
Labels
Milestone

Comments

@evowizz
Copy link

evowizz commented Nov 16, 2023

It looks like Android 14 QPR2 (released yesterday) changed the way APKs are compiled. Apktool is unable to decode the resources.arsc file.

Information

  1. Apktool Version (apktool -version) - 2.9.0 / also tested with a manually built snapshot based on 4441648
  2. Operating System (Mac, Linux, Windows) - Mac
  3. APK From? (Playstore, ROM, Other) - System app, pulled from device
  4. Java Version (java --version) - java 17.0.8 2023-07-18 LTS

Stacktrace

First error:

I: Using Apktool 2.9.0 on example.apk
I: Loading resource table...
Exception in thread "main" brut.androlib.exceptions.AndrolibException: Could not decode arsc file
	at brut.androlib.res.decoder.ARSCDecoder.decode(ARSCDecoder.java:49)
	at brut.androlib.res.data.ResTable.loadResPackagesFromApk(ResTable.java:182)
	at brut.androlib.res.data.ResTable.loadMainPkg(ResTable.java:134)
	at brut.androlib.res.ResourcesDecoder.decodeResources(ResourcesDecoder.java:147)
	at brut.androlib.ApkDecoder.decode(ApkDecoder.java:104)
	at brut.apktool.Main.cmdDecode(Main.java:217)
	at brut.apktool.Main.main(Main.java:92)
Caused by: java.io.IOException: Expected: 0x00000000, got: 0x00000002
	at brut.util.ExtDataInput.skipCheckByte(ExtDataInput.java:56)
	at brut.androlib.res.decoder.ARSCDecoder.readValue(ARSCDecoder.java:469)
	at brut.androlib.res.decoder.ARSCDecoder.readComplexEntry(ARSCDecoder.java:440)
	at brut.androlib.res.decoder.ARSCDecoder.readEntryData(ARSCDecoder.java:375)
	at brut.androlib.res.decoder.ARSCDecoder.readTableType(ARSCDecoder.java:332)
	at brut.androlib.res.decoder.ARSCDecoder.readResourceTable(ARSCDecoder.java:95)
	at brut.androlib.res.decoder.ARSCDecoder.decode(ARSCDecoder.java:44)
	... 6 more

Note

The value 0x00000002 may be different depending on which APK is being decompiled. The APK causing this exact error is within the attached zip.

Steps to Reproduce

  1. apktool d example.apk

Frameworks

The framework-res.apkfile is attached in the zip file.

APK

An example.apk file is attached in the zip file.

Attachments:

qpr2_bug.zip

@iBotPeaches
Copy link
Owner

The heap size error is unrelated. Apps are just getting bigger and Apktool does some things inefficiently, we bumped to 1g here 8477d99

The first error is real and odd. Confirmed.

@iBotPeaches iBotPeaches added this to the v2.9.1 milestone Nov 16, 2023
@iBotPeaches iBotPeaches self-assigned this Nov 16, 2023
@evowizz
Copy link
Author

evowizz commented Nov 16, 2023

Ah I see, thanks. Updating the script fixed what I called the "second error", my bad. Will edit the original comment to only mention the real issue.

@iBotPeaches
Copy link
Owner

Have some time this morning, not to really solve but to research.

➜  3428 java -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005 -jar /usr/local/bin/apktool.jar d example.apk -s -f -v
Listening for transport dt_socket at address: 5005
I: Using Apktool v2.9.0-17-44416481-SNAPSHOT on example.apk
I: Loading resource table...
F: Chunk #1 start=0x00000008 type=0x0002 chunkSize=0x00016f24
F: Chunk #2 start=0x00000014 type=0x0001 chunkSize=0x0000d9e8
F: Chunk #3 start=0x0000d9fc type=0x0200 chunkSize=0x00009530
F: Chunk #4 start=0x0000f638 type=0x0202 chunkSize=0x00000018
F: Chunk #5 start=0x0000f650 type=0x0201 chunkSize=0x00000068
F: Chunk #6 start=0x0000f6b8 type=0x0202 chunkSize=0x00000054
F: Chunk #7 start=0x0000f70c type=0x0201 chunkSize=0x00000290

We crash on chunk 7, so lets dump aapt2 inspection of it.

    [ResTable_type] chunkSize: 656 headerSize: 84 id: 0x02 name: attr flags: 0x02 entryCount: 17 entryStart: 120 config: 
      [ResTable_map_entry] id: 0x0000 name: alpha keyIndex: 2 size: 16 flags: 0x0001 count: 0x0001 parent: 0x00000000
        [Res_value] size: 8 dataType: 0x10 data: 0x00000020 (32)
         name:  name-id:16777216
      [ResTable_map_entry] id: 0x0001 name: font keyIndex: 3 size: 16 flags: 0x0001 count: 0x0001 parent: 0x00000000
        [Res_value] size: 8 dataType: 0x10 data: 0x00000001 (1)
         name:  name-id:16777216
      [ResTable_map_entry] id: 0x0002 name: fontProviderAuthority keyIndex: 4 size: 16 flags: 0x0001 count: 0x0001 parent: 0x00000000
        [Res_value] size: 8 dataType: 0x10 data: 0x00000002 (2)
         name:  name-id:16777216
      [ResTable_map_entry] id: 0x0003 name: fontProviderCerts keyIndex: 5 size: 16 flags: 0x0001 count: 0x0001 parent: 0x00000000
        [Res_value] size: 8 dataType: 0x10 data: 0x00000001 (1)
         name:  name-id:16777216
      [ResTable_map_entry] id: 0x0004 name: fontProviderFetchStrategy keyIndex: 6 size: 16 flags: 0x0001 count: 0x0003 parent: 0x00000000
        [Res_value] size: 8 dataType: 0x10 data: 0x00010000 (65536)
         name:  name-id:16777216
        [Res_value] size: 8 dataType: 0x10 data: 0x00000001 (1)
         name:  name-id:2131099694
        [Res_value] size: 8 dataType: 0x10 data: 0x00000000 (0)
         name:  name-id:2131099695
      [ResTable_map_entry] id: 0x0005 name: fontProviderFetchTimeout keyIndex: 7 size: 16 flags: 0x0001 count: 0x0002 parent: 0x00000000
        [Res_value] size: 8 dataType: 0x10 data: 0x00010004 (65540)
         name:  name-id:16777216
        [Res_value] size: 8 dataType: 0x10 data: 0xffffffff (-1)
         name:  name-id:2131099699
      [ResTable_map_entry] id: 0x0006 name: fontProviderPackage keyIndex: 8 size: 16 flags: 0x0001 count: 0x0001 parent: 0x00000000
        [Res_value] size: 8 dataType: 0x10 data: 0x00000002 (2)
         name:  name-id:16777216
      [ResTable_map_entry] id: 0x0007 name: fontProviderQuery keyIndex: 9 size: 16 flags: 0x0001 count: 0x0001 parent: 0x00000000
        [Res_value] size: 8 dataType: 0x10 data: 0x00000002 (2)
         name:  name-id:16777216
      [ResTable_map_entry] id: 0x0008 name: fontProviderSystemFontFamily keyIndex: 10 size: 16 flags: 0x0001 count: 0x0001 parent: 0x00000000
        [Res_value] size: 8 dataType: 0x10 data: 0x00000002 (2)
         name:  name-id:16777216
      [ResTable_map_entry] id: 0x0009 name: fontStyle keyIndex: 11 size: 16 flags: 0x0001 count: 0x0003 parent: 0x00000000
        [Res_value] size: 8 dataType: 0x10 data: 0x00010000 (65536)
         name:  name-id:16777216
        [Res_value] size: 8 dataType: 0x10 data: 0x00000001 (1)
         name:  name-id:2131099704
        [Res_value] size: 8 dataType: 0x10 data: 0x00000000 (0)
         name:  name-id:2131099707
      [ResTable_map_entry] id: 0x000a name: fontVariationSettings keyIndex: 12 size: 16 flags: 0x0001 count: 0x0001 parent: 0x00000000
        [Res_value] size: 8 dataType: 0x10 data: 0x00000002 (2)
         name:  name-id:16777216
      [ResTable_map_entry] id: 0x000b name: fontWeight keyIndex: 13 size: 16 flags: 0x0001 count: 0x0001 parent: 0x00000000
        [Res_value] size: 8 dataType: 0x10 data: 0x00000004 (4)
         name:  name-id:16777216
      [ResTable_map_entry] id: 0x000c name: lStar keyIndex: 14 size: 16 flags: 0x0001 count: 0x0001 parent: 0x00000000
        [Res_value] size: 8 dataType: 0x10 data: 0x00000020 (32)
         name:  name-id:16777216
      [ResTable_map_entry] id: 0x000d name: nestedScrollViewStyle keyIndex: 15 size: 16 flags: 0x0001 count: 0x0001 parent: 0x00000000
        [Res_value] size: 8 dataType: 0x10 data: 0x00000001 (1)
         name:  name-id:16777216
      [ResTable_map_entry] id: 0x000e name: queryPatterns keyIndex: 16 size: 16 flags: 0x0001 count: 0x0001 parent: 0x00000000
        [Res_value] size: 8 dataType: 0x10 data: 0x00000001 (1)
         name:  name-id:16777216
      [ResTable_map_entry] id: 0x000f name: shortcutMatchRequired keyIndex: 17 size: 16 flags: 0x0001 count: 0x0001 parent: 0x00000000
        [Res_value] size: 8 dataType: 0x10 data: 0x00000008 (8)
         name:  name-id:16777216
      [ResTable_map_entry] id: 0x0010 name: ttcIndex keyIndex: 18 size: 16 flags: 0x0001 count: 0x0001 parent: 0x00000000
        [Res_value] size: 8 dataType: 0x10 data: 0x00000004 (4)
         name:  name-id:16777216

Best I can tell is this sample is reporting as the set of resources are offset16, but in fact are not. You can see this by expanding my search to the surrounding chunks.

    [RES_TABLE_TYPE_SPEC_TYPE] chunkSize: 24 headerSize: 16
    [ResTable_type] chunkSize: 104 headerSize: 84 id: 0x01 name: anim flags: 0x02 entryCount: 2 entryStart: 88 config: 
      [ResTable_entry_compact] id: 0x0000 name: launch_dialog_enter keyIndex: 0 size: 8 flags: 0x0308
        [Res_value] size: 8 dataType: 0x03 data: 0x00000000 ((file) res/anim/launch_dialog_enter.xml type=XML)
      [ResTable_entry_compact] id: 0x0001 name: launch_dialog_exit keyIndex: 1 size: 8 flags: 0x0308
        [Res_value] size: 8 dataType: 0x03 data: 0x00000001 ((file) res/anim/launch_dialog_exit.xml type=XML)
    [RES_TABLE_TYPE_SPEC_TYPE] chunkSize: 84 headerSize: 16
    [ResTable_type] chunkSize: 656 headerSize: 84 id: 0x02 name: attr flags: 0x02 entryCount: 17 entryStart: 120 config: 
      [ResTable_map_entry] id: 0x0000 name: alpha keyIndex: 2 size: 16 flags: 0x0001 count: 0x0001 parent: 0x00000000
        [Res_value] size: 8 dataType: 0x10 data: 0x00000020 (32)
         name:  name-id:16777216
      [ResTable_map_entry] id: 0x0001 name: font keyIndex: 3 size: 16 flags: 0x0001 count: 0x0001 parent: 0x00000000
        [Res_value] size: 8 dataType: 0x10 data: 0x00000001 (1)
         name:  name-id:16777216
    [RES_TABLE_TYPE_SPEC_TYPE] chunkSize: 40 headerSize: 16
    [ResTable_type] chunkSize: 144 headerSize: 84 id: 0x03 name: color flags: 0x02 entryCount: 6 entryStart: 96 config: 
      [ResTable_entry_compact] id: 0x0000 name: androidx_core_ripple_material_light keyIndex: 19 size: 8 flags: 0x1c08
        [Res_value] size: 8 dataType: 0x1c data: 0x1f000000 (#1f000000)
      [ResTable_entry_compact] id: 0x0001 name: androidx_core_secondary_text_default_material_light keyIndex: 20 size: 8 flags: 0x1c08
        [Res_value] size: 8 dataType: 0x1c data: 0x8a000000 (#8a000000)

So as you can see the first chunk has flag 2 (offset16), as well as 2nd and 3rd. However, only the 1st and 3rd are actually packed. I've never seen this before - the chunk is lying on about its packing.

@iBotPeaches
Copy link
Owner

I see Beta 1.1 came out - https://developer.android.com/about/versions/14/release-notes#beta1.1

Interesting to note the changelog item

Fixed an issue that sometimes caused some apps to be removed after a system update was installed and then prevented those apps from being installed again. In other cases, affected apps couldn't be installed on a device even if they hadn't been installed previously.

This seems like malformed applications to me, but want to check the latest beta to confirm.

@evowizz
Copy link
Author

evowizz commented Dec 2, 2023

This seems like malformed applications to me, but want to check the latest beta to confirm.

I got curious too after seeing your message, and tried myself. Sadly, after trying to decompile various apps, including the same app provided in the original issue, the problem remains the same, and I got the exact same exception from apktool.

@iBotPeaches
Copy link
Owner

Okay, I'll have to go slower and see if I misunderstood the spec for compact resources.

@iBotPeaches
Copy link
Owner

iBotPeaches commented Dec 2, 2023

    [ResTable_type] chunkSize: 656 headerSize: 84 id: 0x02 name: attr flags: 0x02 entryCount: 17 entryStart: 120 config: 
      [ResTable_map_entry] id: 0x0000 name: alpha keyIndex: 2 size: 16 flags: 0x0001 count: 0x0001 parent: 0x00000000
        [Res_value] size: 8 dataType: 0x10 data: 0x00000020 (32)
         name:  name-id:16777216

Spent some time today. What I found is we are 2 bytes off prior to reading this map_entry.

Historically in Apktool, we've never relied on entriesStart of the spec as we've always read from one section to another and haven't had a reason to skip/random-read.

  • start - 63236
  • end - 63892
  • size - 656
  • entryStart - chunk start + 120
  • actual location - 63354
  • intended location - 63356

@iBotPeaches
Copy link
Owner

➜  3428 java -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005 -jar /usr/local/bin/apktool.jar d example.apk -s -f
I: Using Apktool v2.9.0-17-44416481-SNAPSHOT on example.apk
I: Loading resource table...
I: Decoding file-resources...
I: Loading resource table from file: /home/ibotpeaches/.local/share/apktool/framework/1.apk
I: Decoding values */* XMLs...
I: Decoding AndroidManifest.xml with resources...
I: Regular manifest package...
I: Copying raw classes.dex file...
I: Copying assets and libs...
I: Copying unknown files...
I: Copying original files...
I: Copying META-INF/services directory
➜  3428 

Not really happy w/ this patch attached, because I'm convinced I'm missing some form of alignment/padding, but I've looked over this for a solid bit - https://cs.android.com/android/platform/superproject/main/+/main:frameworks/base/libs/androidfw/include/androidfw/ResourceTypes.h;l=1476;bpv=0

Don't see anything wrong. So I'll hold this draft PR until source is out for QPR2 for some final inspections.

@evowizz
Copy link
Author

evowizz commented Dec 3, 2023

Mmh, the last time ResourcesTypes.h was updated was in January. I don't think the sources for the QPR2 are live yet. I suppose once the sources will be released, it might be much easier to see what Google changed?

@iBotPeaches
Copy link
Owner

Yep, looking for changes to this even though there isn't really a possible location for anything to change.

/**
 * A collection of resource entries for a particular resource data
 * type.
 *
 * If the flag FLAG_SPARSE is not set in `flags`, then this struct is
 * followed by an array of uint32_t defining the resource
 * values, corresponding to the array of type strings in the
 * ResTable_package::typeStrings string block. Each of these hold an
 * index from entriesStart; a value of NO_ENTRY means that entry is
 * not defined.
 *
 * If the flag FLAG_SPARSE is set in `flags`, then this struct is followed
 * by an array of ResTable_sparseTypeEntry defining only the entries that
 * have values for this type. Each entry is sorted by their entry ID such
 * that a binary search can be performed over the entries. The ID and offset
 * are encoded in a uint32_t. See ResTabe_sparseTypeEntry.
 *
 * There may be multiple of these chunks for a particular resource type,
 * supply different configuration variations for the resource values of
 * that type.
 *
 * It would be nice to have an additional ordered index of entries, so
 * we can do a binary search if trying to find a resource by string name.
 */
struct ResTable_type
{
    struct ResChunk_header header;
    enum {
        NO_ENTRY = 0xFFFFFFFF
    };
    
    // The type identifier this chunk is holding.  Type IDs start
    // at 1 (corresponding to the value of the type bits in a
    // resource identifier).  0 is invalid.
    uint8_t id;
    
    enum {
        // If set, the entry is sparse, and encodes both the entry ID and offset into each entry,
        // and a binary search is used to find the key. Only available on platforms >= O.
        // Mark any types that use this with a v26 qualifier to prevent runtime issues on older
        // platforms.
        FLAG_SPARSE = 0x01,
        // If set, the offsets to the entries are encoded in 16-bit, real_offset = offset * 4u
        // An 16-bit offset of 0xffffu means a NO_ENTRY
        FLAG_OFFSET16 = 0x02,
    };
    uint8_t flags;
    // Must be 0.
    uint16_t reserved;
    
    // Number of uint32_t entry indices that follow.
    uint32_t entryCount;
    // Offset from header where ResTable_entry data starts.
    uint32_t entriesStart;
    // Configuration this collection of entries is designed for. This must always be last.
    ResTable_config config;
};

Since ResTable_config has to be last - its already supported for skipping unread bytes. So I think its just a natural missed feature that entriesStart is finally relevant.

Normally you'd read data and land where entries were as a natural by-product of reading all, so the properties that labeled where data starts isn't relevant to Apktool. Though it appears now maybe that isn't the case, my guess being some alignment.

@evowizz
Copy link
Author

evowizz commented Dec 3, 2023

I can confirm your PR does fix the issue. I built apktool based on your branch and used it to attempt to decompile various apks that previously failed. They were successfully decompiled.

@iBotPeaches
Copy link
Owner

thanks. This was last issue prior to 2.9.1 so let me sleep on this if I go forward with skip patch or hold it.

iBotPeaches added a commit that referenced this issue Dec 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants