listArchive not working #36

tonywoode · 2023-09-16T20:55:09Z

MacOS 10.15.7, node tested from v14-v20.3.0

    try {
      const archivePaths = await listArchive(gamePathMacOS)
      console.log('result is', archivePaths)
    } catch (err) {
      console.log(err)
    }

prints (only)

result is {
  path: '/Volumes/Untitled/Games/Sega Games/Genesis Games/GoodGEN_3_GM/Back to the Future Part III.7z',
  type: '7z',
  physicalSize: 178222,
  headersSize: 311,
  method: 'LZMA:22'
}

expected result would be a list of files in the archive (i.e.: the filenames from the tail of example command-line p7zip output):

$ 7z l /Volumes/Untitled/Games/Sega\ Games/Genesis\ Games/GoodGEN_3_GM/Back\ to\ the\ Future\ Part\ III.7z

7-Zip [64] 17.05 : Copyright (c) 1999-2021 Igor Pavlov : 2017-08-28
p7zip Version 17.05 (locale=utf8,Utf16=on,HugeFiles=on,64 bits,12 CPUs x64)

Scanning the drive for archives:
1 file, 178222 bytes (175 KiB)

Listing archive: /Volumes/Untitled/Games/Sega Games/Genesis Games/GoodGEN_3_GM/Back to the Future Part III.7z

--
Path = /Volumes/Untitled/Games/Sega Games/Genesis Games/GoodGEN_3_GM/Back to the Future Part III.7z
Type = 7z
Physical Size = 178222
Headers Size = 311
Method = LZMA:22
Solid = +
Blocks = 1

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2000-10-29 05:00:00 ....A       524288       177911  Back to the Future Part III (E) [!].gen
2000-10-29 05:00:00 ....A       524288               Back to the Future Part III (U) [!].gen
2000-10-29 05:00:00 ....A       524288               Back to the Future Part III (U) [b1].gen
1996-09-13 13:39:04 ....A       524285               Back to the Future Part III (U) [b2].gen
2001-02-05 06:24:06 ....A       524288               Back to the Future Part III (U) [h1C].gen
1997-11-04 04:46:14 ....A       524288               Back to the Future Part III (U) [h2C].gen
2004-01-01 06:00:00 ....A       524288               Back to the Future Part III (U) [h3C].gen
2004-08-15 23:35:18 ....A       524288               Back to the Future Part III (U) [T+Rus].gen
------------------- ----- ------------ ------------  ------------------------
2004-08-15 23:35:18            4194301       177911  8 files

The text was updated successfully, but these errors were encountered:

tonywoode · 2023-09-16T21:45:21Z

logging entries seems ok:

export const listArchive = (SevenZip.listArchive =
    SevenZip.list =
        function (filepath, options, override = false) {
            return when.promise(function (resolve, reject, progress) {
                let spec = {};
                /* jshint maxlen: 130 */
                let regex = /(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}) ([\.D][\.R][\.H][\.S][\.A]) +(\d+) +(\d+)? +(.+)/;
                /* jshint maxlen: 80 */
                let buffer = ''; //Store incomplete line of a progress data.
                /**
                 * When a stdout is emitted, parse each line and search for a pattern.When
                 * the pattern is found, extract the file (or directory) name from it and
                 * pass it to an array. Finally returns this array.
                 */
                function onprogress(data) {
                    let entries = [];
                    if (buffer.length > 0) {
                        data = buffer + data;
                        buffer = '';
                    }
                    const params = [
                        ['Path = ', 'path', null],
                        ['Type = ', 'type', null],
                        ['Method = ', 'method', null],
                        ['Physical Size = ', 'physicalSize', parseInt],
                        ['Headers Size = ', 'headersSize', parseInt],
                    ];
                    const lines = data.split('\n');
                    for (const line of lines) {
                        let lineDone = false;
                        for (const [beginning, paramType, fn] of params) {
                            if (line.startsWith(beginning)) {
                                const paramValue = line.slice(beginning.length);
                                spec[paramType] = fn ? fn(paramValue) : paramValue;
                                lineDone = true;
                                break;
                            }
                        }
                        if (lineDone)
                            continue;
                        // Parse the stdout to find entries
          console.log(line);
                        let res = regex.exec(line);
                        if (res) {
                            let e = {
                                date: new Date(res[1]),
                                attr: res[2],
                                size: parseInt(res[3], 10),
                                name: ReplaceNativeSeparator(res[5]),
                            };
                            entries.push(e);
                        } // Line may be incomplete, Save it to the buffer.
                        else
                            buffer = line;
                    }
          console.log(entries)
                    return entries;
                }

prints

7-Zip (a) [64] 17.02 : Copyright (c) 1999-2020 Igor Pavlov : 2017-08-28
p7zip Version 17.02 (locale=utf8,Utf16=on,HugeFiles=on,64 bits,12 CPUs x64)

Scanning the drive for archives:
1 file, 178222 bytes (175 KiB)

Listing archive: /Volumes/Untitled/Games/Sega Games/Genesis Games/GoodGEN_3_GM/Back to the Future Part III.7z

--
Solid = +
Blocks = 1

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2000-10-29 05:00:00 ....A       524288       177911  Back to the Future Part III (E) [!].gen
2000-10-29 05:00:00 ....A       524288               Back to the Future Part III (U) [!].gen
2000-10-29 05:00:00 ....A       524288               Back to the Future Part III (U) [b1].gen
1996-09-13 13:39:04 ....A       524285               Back to the Future Part III (U) [b2].gen
2001-02-05 06:24:06 ....A       524288               Back to the Future Part III (U) [h1C].gen
1997-11-04 04:46:14 ....A       524288               Back to the Future Part III (U) [h2C].gen
2004-01-01 06:00:00 ....A       524288               Back to the Future Part III (U) [h3C].gen
2004-08-15 23:35:18 ....A       524288               Back to the Future Part III (U) [T+Rus].gen
------------------- ----- ------------ ------------  ------------------------
2004-08-15 23:35:18            4194301       177911  8 files

[
  {
    date: 2000-10-29T05:00:00.000Z,
    attr: '....A',
    size: 524288,
    name: 'Back to the Future Part III (E) [!].gen'
  },
  {
    date: 2000-10-29T05:00:00.000Z,
    attr: '....A',
    size: 524288,
    name: 'Back to the Future Part III (U) [!].gen'
  },
  {
    date: 2000-10-29T05:00:00.000Z,
    attr: '....A',
    size: 524288,
    name: 'Back to the Future Part III (U) [b1].gen'
  },
  {
    date: 1996-09-13T12:39:04.000Z,
    attr: '....A',
    size: 524285,
    name: 'Back to the Future Part III (U) [b2].gen'
  },
  {
    date: 2001-02-05T06:24:06.000Z,
    attr: '....A',
    size: 524288,
    name: 'Back to the Future Part III (U) [h1C].gen'
  },
  {
    date: 1997-11-04T04:46:14.000Z,
    attr: '....A',
    size: 524288,
    name: 'Back to the Future Part III (U) [h2C].gen'
  },
  {
    date: 2004-01-01T06:00:00.000Z,
    attr: '....A',
    size: 524288,
    name: 'Back to the Future Part III (U) [h3C].gen'
  },
  {
    date: 2004-08-15T22:35:18.000Z,
    attr: '....A',
    size: 524288,
    name: 'Back to the Future Part III (U) [T+Rus].gen'
  }
]

TheTechsTech · 2023-09-17T00:56:06Z

Not sure why that could be happening on Mac.

Have you tried/confirmed on non macOS device?
What version of Node you using?

tonywoode · 2023-09-17T06:06:33Z

Thanks for the quick response, I've tested on MacOS on node's from v14-v20.3.0, and I've just tested on Windows 10 on node 14.16.1 which gives the identical problem. I'll continue to try and poke my way through the code, prob tomorrow eve

tonywoode · 2023-09-18T19:53:48Z

So far, I can't tell you much, but can say:

the CLI implementation of list works fine:

$ ./node_modules/node-7z-archive/cli/list.js /Volumes/Untitled/Games/Sega\ Games/Genesis\ Games/GoodGEN_3_GM/Onslaught.7z
Listing...
[
  {
    date: 2000-03-09T14:16:14.000Z,
    attr: '....A',
    size: 524288,
    name: 'Onslaught (U) [c][!].gen'
  },
  {
    date: 2004-05-26T17:40:14.000Z,
    attr: '....A',
    size: 524288,
    name: 'Onslaught (U) [h1C].gen'
  },
  {
    date: 1997-08-27T07:17:42.000Z,
    attr: '....A',
    size: 524288,
    name: 'Onslaught (U) [h2C].gen'
  },
  {
    date: 2003-12-26T20:18:28.000Z,
    attr: '....A',
    size: 524288,
    name: 'Onslaught (U) [h3C].gen'
  },
  {
    date: 2004-05-26T17:40:14.000Z,
    attr: '....A',
    size: 524288,
    name: 'Onslaught (U) [h4C].gen'
  }
]
List of archive /Volumes/Untitled/Games/Sega Games/Genesis Games/GoodGEN_3_GM/Onslaught.7z done!

I can't easily see what happens with earlier versions, since node-7z-archive < 1.1.5 always gives me

 code: 'ENOENT',
  syscall: 'spawn /Users/twoode/CODE/Scripts/qp-ag-grid/qp-ag-grid-remix/qp-ag-grid-remix/node_modules/node-7z-archive/binaries/darwin/7za',

(and 1.1.6 has the same problem as 1.1.7)

TheTechsTech · 2023-09-18T23:06:30Z

code: 'ENOENT',
syscall: 'spawn /Users/twoode/CODE/Scripts/qp-ag-grid/qp-ag-grid-remix/qp-ag-grid-remix/node_modules/node-7z-archive/binaries/darwin/7za',

That means binary not actually available. Not downloaded and extracted there.

You might need to delete whole folder tree and do another install.

tonywoode · 2023-09-19T21:12:40Z

https://github.com/TheTechsTech/node-7z-archive/blob/master/src/index.ts#L335

Change:

- .progress((data: string) => progress(onprogress(data)))
+ .progress((data) => resolve(onprogress(data)))

listArchive now returns (at least for me in this instance (because the following resolve on the line that follows it now gets ignored)) the array of the list of contents of the archive. However, i'm quite confused! I'm not familiar with using .progress however, specifically:

Given:

                const runner = () => Run(executables[position], command, options, override)
                    .progress((data: string) => progress(onprogress(data)))        
                    .then((args: any) => resolve(position === 0 ? spec : args))

why does the result of .progress not chain onto args of the .then that follows it? args of the .then is the args passed to 7z:

[
  'l',
  '/Volumes/Untitled/Games/Sega Games/Genesis Games/GoodGEN_3_GM/Back to the Future Part III.7z',
  '-ssc',
  '-y',
  '-bb2'
]

.progress is supposed to show the progress of a promise operation, right? So, why does its result not chain onto the subsequent .then?

The docs say, and the variable naming suggests, that this is intentional. https://github.com/TheTechsTech/node-7z-archive/blob/master/src/index.ts#L236

Seems like madness: listArchive's supposed to list the contents of an archive, not show basic archive details, def been like this for some time (I've pasted in previous versions' code going back 5 versions so far). You can, hopefully, understand why I'm confused! Can you help explain? Will my proposed fix work for all cases?

TheTechsTech · 2023-09-19T21:52:12Z

You should read all posted issues about my stand on typescript. That is part of it's introduction by another contributor. Check/run CI tests and submit PR.

tonywoode · 2023-09-22T10:24:31Z

Thanks! I tried to read the posted issues you mention but couldn't find them (are they in node-7z-forall maybe? I'd genuinely be interested if you could point me at the things that happened that make you uncomfortable with the TypeScript code here? Maybe I can offer some help in the future?)

This issue just needs closing: I've just fundamentally misunderstood/failed to grok the API that was set at the beginnning of the node-7z project, since when's .progress hasn't made it down to current Promise-based JS, yet the api of many of the node-7z calls is:

archive.list('archive.zip'
   .progress( files => files ) //allow insertion for progress updates (return is ineffectual)
   .then( spec => spec) //reporting outcome: why you might need the file spec

If only there was an easy way to invert the .progress call into an async/await, whilst still providing a way for progress updates.

Hate to make a bad joke but at least I'm 'making progress' now tho ;-)

TheTechsTech · 2023-09-22T15:12:34Z

... but at least I'm 'making progress' now tho ;-)

No, you have completely drifted off course.

You say, you read issues, still confuse, and think you can help me with typescript. Lol, that's funny!

Which one you did see not #23, nor did go thru PR, https://github.com/TheTechsTech/node-7z-archive/pull/12/files#diff-a2a171449d862fe29692ce031981047d7ab755ae7f84c707aef80701b3ea0c80R74 where typescript was added?

Your solution is just going back to the original https://github.com/TheTechsTech/node-7z-archive/pull/12/files#diff-92bbac9a308cd5fcf9db165841f2d90ce981baddcb2b1e26cfff170929af3bd1

I can go on a rant about peoples adherence to somebody else project code expectation usage.
That's a copier, not any form of programmer, just forget about a actual developer.

Ideas won't evolve, there will be no progress, nothing. So I am going to figure you just brand new to coding.

I have spelled out in some issues that, "I don't actually use JavaScript like when I started the project".

Any real changes will come from others PR's.

tonywoode · 2023-09-22T17:16:48Z

I had seen #23, and in it your comments regarding others' contribution of the TS code, hence me asking you for more info. I hadn't looked through #12 - thanks for the link! I had already appreciated that you don't use JS anymore and only PRs from others will change the project

I wasn't suggesting any solution, just looking to use this module in a project and I'd misread the (somewhat surprising!) API...I shouldn't have opened this issue, the real issue is that the API will likely need changing in the future, and I was mulling on how else to achieve the 2-stage async with-progress the current API provides (as when.promise.tab maybe isn't the right change to put effort into). I could def help with the current TS warnings, but can appreciate my schoolboy-error here might makes you think I couldn't 🤭

tonywoode changed the title ~~listArchive not working on macOS~~ listArchive not working Sep 17, 2023

tonywoode closed this as completed Sep 22, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

listArchive not working #36

listArchive not working #36

tonywoode commented Sep 16, 2023 •

edited

Loading

tonywoode commented Sep 16, 2023 •

edited

Loading

TheTechsTech commented Sep 17, 2023

tonywoode commented Sep 17, 2023 •

edited

Loading

tonywoode commented Sep 18, 2023 •

edited

Loading

TheTechsTech commented Sep 18, 2023

tonywoode commented Sep 19, 2023 •

edited

Loading

TheTechsTech commented Sep 19, 2023

tonywoode commented Sep 22, 2023 •

edited

Loading

TheTechsTech commented Sep 22, 2023

tonywoode commented Sep 22, 2023 •

edited

Loading

listArchive not working #36

listArchive not working #36

Comments

tonywoode commented Sep 16, 2023 • edited Loading

tonywoode commented Sep 16, 2023 • edited Loading

TheTechsTech commented Sep 17, 2023

tonywoode commented Sep 17, 2023 • edited Loading

tonywoode commented Sep 18, 2023 • edited Loading

TheTechsTech commented Sep 18, 2023

tonywoode commented Sep 19, 2023 • edited Loading

TheTechsTech commented Sep 19, 2023

tonywoode commented Sep 22, 2023 • edited Loading

TheTechsTech commented Sep 22, 2023

tonywoode commented Sep 22, 2023 • edited Loading

tonywoode commented Sep 16, 2023 •

edited

Loading

tonywoode commented Sep 16, 2023 •

edited

Loading

tonywoode commented Sep 17, 2023 •

edited

Loading

tonywoode commented Sep 18, 2023 •

edited

Loading

tonywoode commented Sep 19, 2023 •

edited

Loading

tonywoode commented Sep 22, 2023 •

edited

Loading

tonywoode commented Sep 22, 2023 •

edited

Loading