feat: parsing big int values that come from http requests #830

andreabadesso · 2025-03-20T16:01:11Z

Acceptance Criteria

We should be able to parse numbers from wallet-service requests as big ints
We should use zod to validate all responses from the wallet-service lambdas
We should fix all current types that are different from the actual responses

Security Checklist

Make sure you do not include new dependencies in the project unless strictly necessary and do not include dev-dependencies as production ones. More dependencies increase the possibility of one of them being hijacked and affecting us.

codecov · 2025-03-20T16:05:24Z

Codecov Report

Attention: Patch coverage is 84.28571% with 11 lines in your changes missing coverage. Please review.

Project coverage is 82.51%. Comparing base (de961dd) to head (1752d52).
Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
src/wallet/api/walletApi.ts	56.52%	10 Missing ⚠️
src/wallet/wallet.ts	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #830      +/-   ##
==========================================
+ Coverage   81.36%   82.51%   +1.14%     
==========================================
  Files          83       84       +1     
  Lines        6720     6767      +47     
  Branches     1444     1445       +1     
==========================================
+ Hits         5468     5584     +116     
+ Misses       1239     1170      -69     
  Partials       13       13

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

andreabadesso · 2025-03-27T23:28:29Z

src/wallet/types.ts

-  outputs: TxProposalOutputs[]; // Outputs data of the tx proposal
-  tokens: string[];


These are not coming from the wallet-service:

https://github.com/HathorNetwork/hathor-wallet-service/blob/239b5a769b04a09714ca7fa673e99e064e91f471/packages/wallet-service/src/api/txProposalCreate.ts#L138-L145

andreabadesso · 2025-03-27T23:32:40Z

src/wallet/types.ts

@@ -449,14 +454,16 @@ export interface TxByIdTokenData {
  voided: boolean;
  height?: number | null;
  weight: number;
-  balance: Balance;


Type is wrong, it never came as a Balance object from the wallet-service:

https://github.com/HathorNetwork/hathor-wallet-service/blob/239b5a769b04a09714ca7fa673e99e064e91f471/packages/wallet-service/src/db/index.ts#L2963

andreabadesso · 2025-03-27T23:32:53Z

src/wallet/types.ts

  tokenId: string;
-  walletId: string;


Same, no walletId coming from the wallet-service lambdas

src/wallet/api/schemas/walletApi.ts

r4mmer · 2025-03-28T16:30:23Z

src/wallet/api/schemas/walletApi.ts

+ * Schema for token information.
+ */
+export const tokenInfoSchema = z.object({
+  id: z.string(),


Check for 64 length and hex characters?
I included the native token case as well.

Suggested change

id: z.string(),

id: z.string().regex(/^[a-fA-F0-9]{64}$|^00$/),

Great, thanks!

r4mmer · 2025-03-28T16:35:57Z

src/wallet/api/schemas/walletApi.ts

+  rewardSpendMinBlocks: z.number(),
+  maxNumberInputs: z.number(),
+  maxNumberOutputs: z.number(),
+});


Currently any new keys will be ignored, but we can pass them into the final data without validation (so that new keys from the fullnode do not break the api)

Suggested change

});

}).passthrough();

Done!

Also added a test to test for new attributes

r4mmer · 2025-03-31T15:14:13Z

src/wallet/types.ts

+  script: {
+    type: 'Buffer';
+    data: number[];
+  };


Why?
Should the script not be serialized as hex or base64?

Same, can you please check?

r4mmer · 2025-03-31T15:15:10Z

src/wallet/types.ts

+  script: {
+    type: 'Buffer';
+    data: number[];
+  };


If the wallet-service is sending this "Buffer" type maybe we should change how the wallet-service is serializing the script

Hey, we actually receive a simpler TxInput and TxOutput in websocket transactions, e.g.:

{ "height": 0, "inputs": [ { "address": "H79QdDV5i6KuW3dNJEahkUNe8a9axtHuCu", "timelock": null, "type": "P2PKH" } ], "nonce": 16746194, "outputs": [ { "address": "HDY5QHiE33goHcLxFyJmgxTErwgGHDrLdy", "timelock": null, "type": "P2PKH" } ], "parents": [], "signal_bits": 0, "timestamp": 1743779693, "token_name": null, "token_symbol": null, "tx_id": "000009532ec33ba944c5d23b8dc642063539055ca31b4f6d4f534d33651ce8a3", "version": 1, "voided": false, "weight": 16.818364568681208 }

Fixed the type and the schema validation

r4mmer · 2025-03-31T15:20:26Z

src/wallet/connection.ts

-    this.websocket.on('new-tx', payload => this.emit('new-tx', payload.data as WsTransaction));
-    this.websocket.on('update-tx', payload => this.emit('update-tx', payload.data));
+    this.websocket.on('new-tx', payload => {
+      const validatedTx = parseSchema(payload.data, wsTransactionSchema);


Maybe we should put a try..catch around this or use safeParse directly.
Or do you think its better to have an unhandled error than a silent fail (only log an error) on new-tx?

You are right, we shouldn't throw an error here

Fixed

andreabadesso · 2025-04-04T16:04:33Z

src/wallet/wallet.ts

@@ -539,7 +539,7 @@ class HathorWalletServiceWallet extends EventEmitter implements IHathorWallet {
    let shouldGetNewAddresses = false;

    for (const output of outputs) {
-      if (this.newAddresses.find(newAddress => newAddress.address === output.decoded.address)) {


This was a bug, we were not generating new addresses when a new tx came from the websocket.

src/wallet/connection.ts

src/wallet/api/schemas/walletApi.ts

src/wallet/connection.ts

andreabadesso requested a review from pedroferreira1 as a code owner March 20, 2025 16:01

andreabadesso force-pushed the fix/big-int-http-requests branch from b96c062 to c1824f1 Compare March 20, 2025 16:01

andreabadesso self-assigned this Mar 20, 2025

andreabadesso added the enhancement New feature or request label Mar 20, 2025

andreabadesso added this to Hathor Network Mar 20, 2025

github-project-automation bot moved this to Todo in Hathor Network Mar 20, 2025

andreabadesso moved this from Todo to In Progress (Done) in Hathor Network Mar 20, 2025

andreabadesso moved this from In Progress (Done) to In Progress (WIP) in Hathor Network Mar 20, 2025

andreabadesso force-pushed the fix/big-int-http-requests branch 4 times, most recently from eb54782 to 40df312 Compare March 27, 2025 17:22

andreabadesso commented Mar 27, 2025

View reviewed changes

src/wallet/types.ts

tokenId: string;

walletId: string;

Copy link

Contributor Author

andreabadesso Mar 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same, no walletId coming from the wallet-service lambdas

andreabadesso requested a review from r4mmer March 28, 2025 00:20

andreabadesso moved this from In Progress (WIP) to In Progress (Done) in Hathor Network Mar 28, 2025

r4mmer reviewed Mar 28, 2025

View reviewed changes

andreabadesso requested a review from r4mmer March 31, 2025 13:50

andreabadesso force-pushed the fix/big-int-http-requests branch from cdfa371 to 8adf25b Compare March 31, 2025 14:08

r4mmer reviewed Mar 31, 2025

View reviewed changes

andreabadesso force-pushed the fix/big-int-http-requests branch from 8357e31 to fe1b9fb Compare April 4, 2025 16:03

andreabadesso commented Apr 4, 2025

View reviewed changes

andreabadesso requested a review from r4mmer April 4, 2025 16:05

andreabadesso removed the status in Hathor Network Apr 4, 2025

andreabadesso added 4 commits April 4, 2025 13:17

feat: added schemas for all api requests

c704c99

feat: use schemas in wallet api

ec4ae38

fix: AddressInfoObject had an invalid type

a7123a2

refactor: removed unused improt

1c79614

andreabadesso moved this to In Progress (Done) in Hathor Network Apr 4, 2025

r4mmer previously approved these changes Apr 7, 2025

View reviewed changes

andreabadesso dismissed r4mmer’s stale review via 027cfbd April 8, 2025 20:14

tests: added tests to websocket schema

e6d5a2d

andreabadesso force-pushed the fix/big-int-http-requests branch from 027cfbd to e6d5a2d Compare April 8, 2025 20:18

andreabadesso requested a review from r4mmer April 8, 2025 20:51

r4mmer previously approved these changes Apr 8, 2025

View reviewed changes

pedroferreira1 reviewed Apr 9, 2025

View reviewed changes

refactor: created reusable method to handle ws messags

8eff5e0

andreabadesso dismissed r4mmer’s stale review via 8eff5e0 April 10, 2025 13:00

andreabadesso added 2 commits April 10, 2025 10:28

feat: added decimalPlaces and genesis*

c1607e8

refactor: using proper schema for token id

1ffd9ab

andreabadesso moved this from In Progress (Done) to In Progress (WIP) in Hathor Network Apr 10, 2025

refactor: correct schema for fullnode inputs and outputs

90b72dd

andreabadesso force-pushed the fix/big-int-http-requests branch from 2e0f831 to 91fa491 Compare April 10, 2025 18:34

refactor: using AddressSchema for addresses

d09df09

andreabadesso force-pushed the fix/big-int-http-requests branch from 91fa491 to d09df09 Compare April 10, 2025 18:43

refactor: new tx id schema and using it everywhere

ecd92f3

andreabadesso requested review from pedroferreira1 and r4mmer April 10, 2025 21:53

pedroferreira1 previously approved these changes Apr 11, 2025

View reviewed changes

src/wallet/connection.ts Outdated Show resolved Hide resolved

pedroferreira1 moved this from In Progress (WIP) to In Review (WIP) in Hathor Network Apr 11, 2025

feat: logging ws message type

1752d52

andreabadesso dismissed pedroferreira1’s stale review via 1752d52 April 11, 2025 12:08

r4mmer approved these changes Apr 11, 2025

View reviewed changes

pedroferreira1 approved these changes Apr 11, 2025

View reviewed changes

andreabadesso merged commit 13cf37e into master Apr 11, 2025
5 checks passed

github-project-automation bot moved this from In Review (WIP) to Waiting to be deployed in Hathor Network Apr 11, 2025

andreabadesso deleted the fix/big-int-http-requests branch April 11, 2025 18:49

andreabadesso moved this from Waiting to be deployed to Done in Hathor Network Apr 17, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: parsing big int values that come from http requests #830

feat: parsing big int values that come from http requests #830

andreabadesso commented Mar 20, 2025 •

edited

Loading

codecov bot commented Mar 20, 2025 •

edited

Loading

andreabadesso Mar 27, 2025

andreabadesso Mar 27, 2025

andreabadesso Mar 27, 2025

r4mmer Mar 28, 2025

andreabadesso Mar 31, 2025

r4mmer Mar 28, 2025

andreabadesso Mar 31, 2025

r4mmer Mar 31, 2025

andreabadesso Apr 4, 2025

r4mmer Mar 31, 2025

andreabadesso Apr 4, 2025

r4mmer Mar 31, 2025

andreabadesso Apr 4, 2025

andreabadesso Apr 4, 2025

		outputs: TxProposalOutputs[]; // Outputs data of the tx proposal
		tokens: string[];

	id: z.string(),
	id: z.string().regex(/^[a-fA-F0-9]{64}$\|^00$/),

feat: parsing big int values that come from http requests #830

feat: parsing big int values that come from http requests #830

Conversation

andreabadesso commented Mar 20, 2025 • edited Loading

Acceptance Criteria

Security Checklist

codecov bot commented Mar 20, 2025 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andreabadesso commented Mar 20, 2025 •

edited

Loading

codecov bot commented Mar 20, 2025 •

edited

Loading