Skip to content

Commit 721b5b6

Browse files
Downgrade ElasticSearch batch size from 256mib to 32mib (#11752)
* Update ElasticsearchAirbyteMessageConsumerFactory.java * add changleog and bump connector version seed file Co-authored-by: marcosmarxm <[email protected]>
1 parent 2293f89 commit 721b5b6

File tree

5 files changed

+12
-5
lines changed

5 files changed

+12
-5
lines changed

airbyte-config/init/src/main/resources/seed/destination_definitions.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -93,7 +93,7 @@
9393
- destinationDefinitionId: 68f351a7-2745-4bef-ad7f-996b8e51bb8c
9494
name: ElasticSearch
9595
dockerRepository: airbyte/destination-elasticsearch
96-
dockerImageTag: 0.1.1
96+
dockerImageTag: 0.1.2
9797
documentationUrl: https://docs.airbyte.io/integrations/destinations/elasticsearch
9898
icon: elasticsearch.svg
9999
releaseStage: alpha

airbyte-config/init/src/main/resources/seed/destination_specs.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1351,7 +1351,7 @@
13511351
supported_destination_sync_modes:
13521352
- "overwrite"
13531353
- "append"
1354-
- dockerImage: "airbyte/destination-elasticsearch:0.1.1"
1354+
- dockerImage: "airbyte/destination-elasticsearch:0.1.2"
13551355
spec:
13561356
documentationUrl: "https://docs.airbyte.io/integrations/destinations/elasticsearch"
13571357
connectionSpecification:

airbyte-integrations/connectors/destination-elasticsearch/Dockerfile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,5 +16,5 @@ ENV APPLICATION destination-elasticsearch
1616

1717
COPY --from=build /airbyte /airbyte
1818

19-
LABEL io.airbyte.version=0.1.1
19+
LABEL io.airbyte.version=0.1.2
2020
LABEL io.airbyte.name=airbyte/destination-elasticsearch

airbyte-integrations/connectors/destination-elasticsearch/src/main/java/io/airbyte/integrations/destination/elasticsearch/ElasticsearchAirbyteMessageConsumerFactory.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@
2929
public class ElasticsearchAirbyteMessageConsumerFactory {
3030

3131
private static final Logger log = LoggerFactory.getLogger(ElasticsearchAirbyteMessageConsumerFactory.class);
32-
private static final int MAX_BATCH_SIZE_BYTES = 1024 * 1024 * 1024 / 4; // 256mib
32+
private static final int MAX_BATCH_SIZE_BYTES = 1024 * 1024 * 32; // 32mib
3333
private static final ObjectMapper mapper = new ObjectMapper();
3434

3535
private static final AtomicLong recordsWritten = new AtomicLong(0);

docs/integrations/destinations/elasticsearch.md

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,4 +66,11 @@ The connector should be enhanced to support variable batch sizes.
6666
### Setup guide
6767

6868
Enter the hostname and/or other configuration information ...
69-
#### TODO: more info, screenshots?, etc...
69+
## CHANGELOG
70+
71+
| Version | Date | Pull Request | Subject |
72+
| :--- | :--- | :--- | :--- |
73+
| 0.1.2 | 2022-04-19 | [11752](https://github.com/airbytehq/airbyte/pull/11752) | Reduce batch size to 32Mb |
74+
| 0.1.1 | 2022-02-10 | [10256](https://github.com/airbytehq/airbyte/pull/1256) | Add ExitOnOutOfMemoryError connectors |
75+
| 0.1.0 | 2021-10-13 | [7005](https://github.com/airbytehq/airbyte/pull/7005) | Initial release. |
76+

0 commit comments

Comments
 (0)