[improve][broker] Make maxBatchDeletedIndexToPersist configurable and document other related configs

[lhotari]

Motivation

maxBatchDeletedIndexToPersist has been hard coded to 10000 in Pulsar code base. It's better to make this configurable and also document the setting properly in broker.conf.
managedLedgerPersistIndividualAckAsLongArray setting was also missing from broker.conf and it's existing documentation in ServiceConfiguration.java didn't explain the impact to end users.

Modifications

• make maxBatchDeletedIndexToPersist by adding managedLedgerMaxBatchDeletedIndexToPersist setting
• add managedLedgerPersistIndividualAckAsLongArray to broker.conf
• improve documentation for all of these settings

Additional Context

• Another PR [improve][broker] Add managedCursor/LedgerInfoCompressionType settings to broker.conf #24391 adds managedCursorInfoCompressionType and managedLedgerInfoCompressionType settings to broker.conf. managedCursorInfoCompressionType is also related to persisting acknowledgement state.

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 74.23%. Comparing base (bbc6224) to head (a831dbc).
Report is 1132 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #24392      +/-   ##
============================================
+ Coverage     73.57%   74.23%   +0.65%     
- Complexity    32624    32660      +36     
============================================
  Files          1877     1867      -10     
  Lines        139502   145281    +5779     
  Branches      15299    16612    +1313     
============================================
+ Hits         102638   107843    +5205     
+ Misses        28908    28892      -16     
- Partials       7956     8546     +590     

Flag	Coverage Δ
inttests	26.65% <100.00%> (+2.07%)	⬆️
systests	23.29% <100.00%> (-1.03%)	⬇️
unittests	73.71% <100.00%> (+0.87%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...apache/bookkeeper/mledger/ManagedLedgerConfig.java	96.55% <100.00%> (+0.25%)	⬆️
...org/apache/pulsar/broker/ServiceConfiguration.java	98.16% <100.00%> (-1.23%)	⬇️
...rg/apache/pulsar/broker/service/BrokerService.java	83.83% <100.00%> (+3.05%)	⬆️

... and 1078 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Denovo1998]

LGTM!

When the contents of #24391 and #24392 are finalized and merged, I will update again the description of these configurations in PIP-423.

[lhotari]

This has been cherry-picked to maintenance branches, with a change where managedLedgerPersistIndividualAckAsLongArray defaults to false.