6.0 major version upgrade #840

saimaz · 2018-05-03T11:01:04Z

No description provided.

saimaz · 2018-05-03T11:05:15Z

Here's some sneak peak for the configuration:

ongr_elasticsearch:
    _defaults:
        number_of_shards: 4
        refresh_interval: -1
        number_of_replicas: 0
        hosts:
            - 127.0.0.1:9200
    analysis:
        filter:
            incremental_filter: #-> analyzer name
                type: edge_ngram
                min_gram: 1
                max_gram: 20
        analyzer:
            incrementalAnalyzer: #-> analyzer name
                type: custom
                tokenizer: standard
                filter:
                    - lowercase
                    - incremental_filter
    indexes:
        ongr-product-test:
            settings:
                number_of_shards: 2 #-> default 4, and override with 2
            mapping: TestBundle\Document\Product

saimaz · 2018-05-03T11:09:17Z

I have some idea also to make configuration as less as possible. In that case, maybe would make sense to set an index name in the @Document annotation. There would be additional options like: index and index_settings.

…uments (#857)

…n't tested PHP 7.4 and 8.0 yet. (#858)

* fixed php 7.3 compatibility * allow php 7.3

alexander-schranz · 2019-01-08T22:27:48Z

Service/Manager.php

-/**
- * Manager class.
- */
-class Manager


@saimaz this is a big change is this really needed for ES6? This will make it impossible to support ES5 and ES6 at the same time.

@alexander-schranz I wrote a comment in the PR regarding this and all the changes.

saimaz · 2019-01-09T08:15:39Z

It's a prototype. With the 6th version and further development, I don't see the benefits of keeping bc with the 5th version and sf 3. There are so many changes in the symfony and elastic. The only downside is in the project you have to upgrade it. During the time and using this bundle I learned a lot and found some logic mistakes which I want to fix. Also, the intention for these changes is related to two other very useful bundle development and new major version support, the FilterManagerBundle and RouterBundle.

alexander-schranz · 2019-01-09T08:47:47Z

@saimaz

Maybe would make sense to set an index name in the @document annotation

As index name likes database name mostly come from the parameters.yml or env variable I think this is not a good idea to move the index_name to the Document Annotation. At least a index_prefix_name should be configureable in the configuration.

The only downside is in the project you have to upgrade it

For a project its always easy to upgrade as you can make there breaking changes but for a reusable bundle its not.

* Fix profiler view for symfony 4 * Fix travis.yml * Fix tests and add var folder to ignore in travis

alexander-schranz · 2019-01-19T21:33:25Z

@saimaz If we don't have a manager how do we choose where to index a document if you look at our current configuration we are indexing the same document into 2 indexes (based on some business logic its not always indexed in live index). How I could achieve this without having manager as normally you get the correct repository from the manager.

saimaz · 2019-01-21T19:31:26Z

The thing is that from the 6th version there is no need to have a manager anymore. The manager was to support the index with more than one type. From now on every type is an index. I don't want to introduce BC changes so will keep the manager with the existing functions, but will mark it as deprecated and will remove later on in 7th or 8th version.

For the configuration, I imagine that BC can be kept as well. With some additional checks should be everything fine. But later on also will be removed and everything would be great to keep as clean as possible. This will let to work more easily with Symfony auto wiring and other new features.

alexander-schranz · 2019-01-22T15:34:40Z

How you can index the same document in 2 different indexes based on business logic then?

Example configuration:

    indexes:
        admin-index:
            settings:
                number_of_shards: 2 #-> default 4, and override with 2
            mapping: TestBundle\Document\Page
        website-index:
            settings:
                number_of_shards: 2 #-> default 4, and override with 2
            mapping: TestBundle\Document\Page

So there need to be a index service or a paramter on the repository in which index the document should be stored?

saimaz · 2019-01-22T16:12:08Z

The repository would be index representation. So you simply persist to the specific repository which works only with the one index.

* remove collection * update composer dependencies * upgrade tests for v6 elastic * added app kernel auto loading for the tests * improve the container loading * add ignore for the test cache * introducing index annotation and service * introducing index configuration, still experimental * marking the deprecations * update index service by adding more functions * update changelog * expand event variables * mark deprecations * remove deprecated service naming, use namespaces instead * mark deprecations * update * file reader included * changed events and improved mapping pass * cleanup * introduce new metadata collector for the index * starto to work with the object conversion * make first test run * finished index document metadata loading and index service loading * making dependency injection layer for the bundle more stable * index creation * index creation with analysis data works * simple document insert and find works * add more test cases to test iteration and find methods * introduce custom normalizer to support embedded objects in the document * added embedded objects lazy loading * make almost everything working again * fixed code style issues * add clover coverage to the ignore list * missing trailing slash in the beginning * update travis build requirement for version matrix

…nctional testing

chirimoya · 2019-03-19T11:49:55Z

@saimaz can you give us a short update on the state of this PR? thx

saimaz · 2019-03-19T11:55:19Z

This PR will be closed in favor of #868. It was my experimental version of what it could be and to ask community suggestions.

You can check the 6.0-dev branch and functional tests on how the usage has been changed. There are not so many changes but rather complexity reduces. Full documentation, change list and bc changes, upgrade info are coming :) I didn't write it in a first place since I want to fully test the prototype which I released as an early 6.0.0-beta.

toooni · 2019-03-21T10:51:24Z

DependencyInjection/Compiler/MappingPass.php

-                    [$repositoryDetails['namespace']]
-                );
-                $repositoryDefinition->setPublic(true);
+        $defaultIndex = $defaultIndex ?? array_shift(array_keys($indexes));


If no index is present (f.e. in a new environment) array_shift(array_keys($indexes)) is invalid.

That is completely true. Thanks for the notice.

toooni · 2019-03-21T11:07:55Z

Mapping/DocumentParser.php

    {
-        $result = $this->reader->getPropertyAnnotation($property, self::EMBEDDED_ANNOTATION);
+        $class = new \ReflectionClass($namespace);


We should catch ReflectionException here. Filenames / Classes do not always match.

try { $class = new \ReflectionClass($namespace); } catch (\ReflectionException $e) { return []; }

Well if filenames and classes (namespaces) won't match it will break PSR-4, and composer will throw an exception. Anyway, thanks for the insight, will think about how to improve this place.

No. You are iterating over the users project dir. You don't know whats hidden in there ;)
In my case it was a wrong defined test namespace. This isn't a static analysis tool.

Ok, I misunderstood for the first time. The idea behind this is to iterate through the src directory and collect the files which have @Index annotation which indicates that is an ElasticSearch index. THen it grabs the namespace from that file and loads the reflection class. I agree that the exception has to be thrown if it cannot load it.

I saw that, yes. I wonder if there's another way which might be a bit more performant?
Do you think it might be too complicated for the users to just allow a specific folder like Document?

I was thinking about a more performant way to get the mapping. But it will be executed once on warmup and then cached. So not a big deal if it will take 1-second longer IMO.

Regarding the additional categories, I already have introduced a specific parameter for that.

Take a look at the #869 It's the latest state there and potential RC version which most likely won't change drastically.

Sorry the Document folder thing was related to the performance. I meant to require all Documents to be inside a Document folder. This way not all classes are checked. But as long as it's really just a second - I think you're right.

saimaz · 2019-03-21T11:59:50Z

I'm closing this branch in favor of #868 and v6.0.0-beta release. Will open a pull request to the master from 6.0-dev where currently is the latest state of a 6th major version upgrade. If you anyone noticed something wrong or have any idea to improve any place, please comment it out there.

Simonas Šerlinskas added 5 commits May 3, 2018 13:42

renamed object and nested annotations

ff34065

added support for phpunit 6

41a9177

update composer

cc93c87

small updates and fixes

283ad5e

update travis

d30af19

saimaz mentioned this pull request May 3, 2018

Symfony 4 compatibility #831

Closed

Simonas Šerlinskas added 2 commits May 3, 2018 16:37

introduce promise version of the configuration

3c54b37

first steps for the repository only approach

1ef8129

saimaz mentioned this pull request Jun 19, 2018

Object is reserved word in PHP 7.2 #828

Closed

jcerri and others added 4 commits November 15, 2018 18:31

Handle ObjectType & replace deprecated Collection when generating doc…

c35e1bc

…uments (#857)

Restrict compatible PHP versions to currently supported ones. We have…

7aa6fb3

…n't tested PHP 7.4 and 8.0 yet. (#858)

Merge remote-tracking branch 'remotes/origin/master' into 5.2

c65b444

Fixed php 7.3 compatibility (#862)

482957a

* fixed php 7.3 compatibility * allow php 7.3

alexander-schranz reviewed Jan 8, 2019

View reviewed changes

Kifir22 and others added 3 commits January 10, 2019 19:21

Events::PRE_PERSIST work with object before bulk (#866)

d27b57e

Fix profiler view for symfony 4 (#863)

15cbc8d

* Fix profiler view for symfony 4 * Fix travis.yml * Fix tests and add var folder to ignore in travis

Use modified PRE_MANAGER_CREATE indexSettings (#834)

1689977

Simonas Šerlinskas and others added 5 commits February 18, 2019 15:58

fix manager factory test case with missing tracer pass

88605bc

allow to use elasticsearch 5 or 6

fef6acb

added cache dir to the ignored list

11b116e

Merge 5.2 branch to 6.0-dev

191b00d

Simonas Šerlinskas added 4 commits March 19, 2019 11:34

add more information to the config example

505606c

remove not necessary cache

6dcb860

dependency injection is a symfony stuff, so we can coverage it via fu…

0ea826b

…nctional testing

update the readme

9fe0fa4

toooni reviewed Mar 21, 2019

View reviewed changes

saimaz closed this Mar 21, 2019

demianchuk mentioned this pull request Aug 4, 2017

How to get results of inner hits in ongr? #808

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

6.0 major version upgrade #840

6.0 major version upgrade #840

saimaz commented May 3, 2018

saimaz commented May 3, 2018

saimaz commented May 3, 2018

alexander-schranz Jan 8, 2019

saimaz Jan 9, 2019

saimaz commented Jan 9, 2019

alexander-schranz commented Jan 9, 2019

alexander-schranz commented Jan 19, 2019

saimaz commented Jan 21, 2019

alexander-schranz commented Jan 22, 2019

saimaz commented Jan 22, 2019 •

edited

Loading

chirimoya commented Mar 19, 2019

saimaz commented Mar 19, 2019

toooni Mar 21, 2019

saimaz Mar 21, 2019

toooni Mar 21, 2019

saimaz Mar 21, 2019

toooni Mar 21, 2019

saimaz Mar 21, 2019

toooni Mar 21, 2019

saimaz Mar 21, 2019

toooni Mar 21, 2019 •

edited

Loading

saimaz commented Mar 21, 2019

6.0 major version upgrade #840

6.0 major version upgrade #840

Conversation

saimaz commented May 3, 2018

saimaz commented May 3, 2018

saimaz commented May 3, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

saimaz commented Jan 9, 2019

alexander-schranz commented Jan 9, 2019

alexander-schranz commented Jan 19, 2019

saimaz commented Jan 21, 2019

alexander-schranz commented Jan 22, 2019

saimaz commented Jan 22, 2019 • edited Loading

chirimoya commented Mar 19, 2019

saimaz commented Mar 19, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

toooni Mar 21, 2019 • edited Loading

Choose a reason for hiding this comment

saimaz commented Mar 21, 2019

saimaz commented Jan 22, 2019 •

edited

Loading

toooni Mar 21, 2019 •

edited

Loading