Skip to content

implement sync formdata parser#2911

Merged
mcollina merged 11 commits into
nodejs:mainfrom
KhafraDev:formdata-parsing-x2
Mar 5, 2024
Merged

implement sync formdata parser#2911
mcollina merged 11 commits into
nodejs:mainfrom
KhafraDev:formdata-parsing-x2

Conversation

@KhafraDev

@KhafraDev KhafraDev commented Mar 3, 2024

Copy link
Copy Markdown
Member

Fixes #2890

Based on https://andreubotella.github.io/multipart-form-data/

import { Request } from 'undici'

const request = new Request('http://localhost', {
  method: 'POST',
  headers: {
    'Content-Type': 'multipart/form-data; boundary=----formdata-undici-0.6204674738279623'
  },
  body:
    '------formdata-undici-0.6204674738279623\r\n' +
    'Content-Disposition: form-data; name="fiŝo"\r\n' +
    '\r\n' +
    'value1\r\n' +
    '------formdata-undici-0.6204674738279623--'
})

const fd = await request.formData()
// > FormData { [Symbol(state)]: [ { name: 'fiŝo', value: 'value1' } ] }

Comment thread lib/web/fetch/formdata-parser.js Fixed
Comment thread lib/web/fetch/formdata-parser.js Fixed
Comment thread lib/web/fetch/formdata-parser.js Fixed
@codecov-commenter

codecov-commenter commented Mar 3, 2024

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 83.51449% with 91 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.62%. Comparing base (fc4e766) to head (2f69f5d).
⚠️ Report is 997 commits behind head on main.

Files with missing lines Patch % Lines
lib/web/fetch/formdata-parser.js 80.67% 91 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2911      +/-   ##
==========================================
- Coverage   93.86%   93.62%   -0.25%     
==========================================
  Files          85       86       +1     
  Lines       23418    23861     +443     
==========================================
+ Hits        21981    22339     +358     
- Misses       1437     1522      +85     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@KhafraDev

Copy link
Copy Markdown
Member Author

Alternatively we could just take Deno's parser.

Comment thread lib/web/fetch/formdata-parser.js Fixed
Comment thread lib/web/fetch/formdata-parser.js Fixed
Comment thread lib/web/fetch/formdata-parser.js Fixed
Comment thread lib/web/fetch/formdata-parser.js Outdated
@KhafraDev KhafraDev marked this pull request as ready for review March 3, 2024 15:53
Comment thread package.json
Comment thread lib/web/fetch/body.js
Comment thread lib/web/fetch/body.js

@mcollina mcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollina

mcollina commented Mar 3, 2024

Copy link
Copy Markdown
Member

Does this increase our wpt conpliance?

Comment thread lib/web/fetch/body.js
@KhafraDev

KhafraDev commented Mar 3, 2024

Copy link
Copy Markdown
Member Author

Does this increase our wpt conpliance?

Yes, there are 4 (out of 14 tests wrt .formData) that now pass and there are some untested behaviors/behaviors we had to workaround that are now correct:

  • We actually follow the steps in the spec.
  • .formData() is now a function that returns a Promise instead of being async (I asked in the WHATWG matrix chat and this is the expected behavior)
  • We were able to get rid of hacks in the code for a WPT: "Fetch aborted & connection closed when aborted after calling response.formData()" in fetch/api/abort/general.any.js
        // Wait a tick before checking if the request has been aborted.
        // Otherwise, a TypeError can be thrown when an AbortError should.
        await Promise.resolve()

Comment thread lib/web/fetch/util.js
Comment thread lib/web/fetch/util.js
Comment thread lib/web/fetch/formdata-parser.js Outdated
@Uzlopak

This comment was marked as off-topic.

Comment thread lib/web/fetch/formdata-parser.js
Comment thread lib/web/fetch/formdata-parser.js
Comment thread lib/web/fetch/formdata-parser.js
Comment thread lib/web/fetch/formdata-parser.js
Comment thread lib/web/fetch/formdata-parser.js Outdated
Comment thread lib/web/fetch/formdata-parser.js Outdated
Comment thread lib/web/fetch/formdata-parser.js Outdated
@KhafraDev

Copy link
Copy Markdown
Member Author

@Uzlopak ptal

@Uzlopak Uzlopak left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@KhafraDev

Copy link
Copy Markdown
Member Author

We can either land this as-is or I can spend the next few days/week (its midterm week for me so it might take longer) porting the busboy tests and fixing edge cases.

@Uzlopak

Uzlopak commented Mar 5, 2024

Copy link
Copy Markdown
Contributor

how about a new branch and port the tests, and then when they are ported we merge it into main?

Can you port a test from busboy so that i can assess how much work it is?

@KhafraDev

KhafraDev commented Mar 5, 2024

Copy link
Copy Markdown
Member Author

Adding the tests is very simple for the sole reason that they don't use any test runner and rely completely on node:assert. I ported the tests in my old formdata parser branch: https://github.com/KhafraDev/undici/tree/undici-formdata-parser/test/busboy

However, I don't think there's much benefit in porting something without also fixing bugs.

@KhafraDev

Copy link
Copy Markdown
Member Author

It might be a little harder, given that we don't implement any limits, nor does the parser extend EventEmitter much less emit events, but it shouldn't take more than an hour?

@Uzlopak

Uzlopak commented Mar 5, 2024

Copy link
Copy Markdown
Contributor

ah you mean the original busboy.

@KhafraDev

Copy link
Copy Markdown
Member Author

I assumed that @fastify/busboy would have the same tests, guess I was wrong. Busboy's tests seem more relevant here since they test the actual parsing/output, where as fastify/busboy seems to test more so the actual api (which obviously we'd fail).

@Uzlopak

Uzlopak commented Mar 5, 2024

Copy link
Copy Markdown
Contributor

actually busboy (original) mixed dicer and co into it. @fastify/busboy vendored them.

Anyhow i think that our @fastify/busboy tests were kind of more reasonable.

What about these tests?

https://github.com/fastify/busboy/blob/master/test/types-multipart.test.js

@KhafraDev

Copy link
Copy Markdown
Member Author

No preference, but I don't like how expected parts removed the objects for arrays, now I have no idea what each index means.

@mcollina mcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@gurgunday gurgunday left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me

@mcollina mcollina merged commit 5bb84ba into nodejs:main Mar 5, 2024
@KhafraDev KhafraDev deleted the formdata-parsing-x2 branch March 5, 2024 17:30
@coderabbitai coderabbitai Bot mentioned this pull request Jun 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

undici parses FormData bodies incorrectly

9 participants