S3 Driver

[bonswouar]

_Don't know why #120 has been closed, those S3 driver problems should be fixed, no ?_

I was first using the Flysystem config for S3 :

                        driver: Flysystem
                        url: %aws.content.url%
                        flysystem:
                            type: aws_s3_v2
                            options:
                                aws_s3_v2:
                                    key: %aws.key%
                                    secret: %aws.secret%
                                    region: %aws.region%
                                    bucket_name: %aws.content.bucket%

That seemed to work fine.. But then I noticed folders aren't visible ! (though creating them work, but they won't be displayed in ElFinder explorer)

---

So I tried using the "native" elFinder S3 driver, here is the (apparently) correct config :

                        driver: S3
                        s3_settings:
                            access_key: %aws.key%
                            secret_key: %aws.secret%
                            bucket: %aws.content.bucket%
                            tmp_path: /tmp

• I first had, as mentioned in the other issue :

  E_COMPILE_ERROR: Declaration of FM\ElFinderPHP\Driver\ElFinderVolumeS3::_save() must be compatible with FM\ElFinderPHP\Driver\ElFinderVolumeDriver::_save($fp, $dir, $name, $stat) in /home/www/git/website.com/vendor/helios-ag/fm-elfinder-php-connector/src/Driver/ElFinderVolumeS3.php line 15

• If I fix that I've got :

  Error: Class 'FM\ElFinderPHP\Driver\SoapClient' not found in /home/www/git/website.com/vendor/helios-ag/fm-elfinder-php-connector/src/Driver/ElFinderVolumeS3.php line 662

• If I fix that I've got :

  {"error":["errConf","errNoVolumes"],"debug":["Driver "FM\ElFinderPHP\Driver\ElFinderVolumeS3" : Path undefined."]}

• I assumed this path is the "tmp_path", so I fixed that and finally I've got a SOAP error :

  Please re-send this request to the specified temporary endpoint. Continue to use the original request endpoint for future requests.
  in /home/www/git/website.com/vendor/helios-ag/fm-elfinder-php-connector/src/Driver/ElFinderVolumeS3.php at line 710

Is anybody able to properly use this Bundle with S3 ? Any help would be appreciated.

=> Update : Second part deprecated in dev-master, now it's using the AWS SDK instead of the SOAP client.

[bonswouar]

Update : Apparently some changes have been made recently on the master branch (maybe that would be nice to add a tag, and update the bundle dependencies ?).

So if I use the updated code of the driver, and use my fix for the "tmp_path", I've got a brand new error :

The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.
in /home/www/git/website.com/vendor/aws/aws-sdk-php/src/Aws/Common/Exception/NamespaceExceptionFactory.php at line 91

Looks like a aws sdk version problem ! But that's weird as it was working previously..

[helios-ag]

Hi @bonswouar, there was a PR helios-ag/ElFinderPHP@aadea4b, that have reworked some behavior i think, and Soap client was removed.

[bonswouar]

Thanks for your answer @helios-ag .

Well, the new question included this PR.

• I still don't completely understand what the path (from the error Path undefined.) is for.
  Anyway it's still missing in the helios-ag/FMElfinderBundle : I guessed it should be $settings['path'] instead of $settings['tmpPath'] in https://github.com/helios-ag/FMElfinderBundle/blob/master/Configuration/ElFinderConfigurationReader.php#L263 .
• So like I said, after fixing that I got the Please use AWS4-HMAC-SHA256 error.
  I searched a bit and found out this is apparently because of the new Signature V4 signing process (which is the only one available on new regions, like the one I'm using eu-central-1).
  It's supposed to detect the signing version from the region, so I tried to add the region to the S3Client::factory() parameters, which seems to be the right way to fix that.

---

But I'm still having problems, apparently because of this path parameter (tmp_path in the config) !

• With upload (a valid S3 root folder) as path that gave me another error :

  404 Not Found (Request-ID: CEFFBB575038BD6B)

• And if I set / as the path I get :

  {"error":["errOpen","default folder","errFolderNotFound"],"debug":{"connector":"php","phpver":"5.6.9-0+deb8u1","time":0.27798795700073,"memory":"65773Kb \/ 60273Kb \/ 512M","upload":"","volumes":[{"id":"s3s1_","name":"\\driver\\elfindervolumes3","mimeDetect":"internal","imgLib":"gd"}],"mountErrors":[]}}

• And finally, if I set /tmp as path.. It first seems to work, but when I check the finder view is empty, having tmp as a root.
  And if I try to create a new folder for example, I get the error The specified bucket does not exist, and I can see that it's trying to concatenate tmp to my Bucket.
  For example when trying to create a folder called ddd for a bucket called mybucket it's calling Aws\S3\StreamWrapper ->createPseudoDirectory with 's3://mybuckettmp/ddd/', array('Bucket' => 'mybuckettmp', 'Key' => 'ddd/', 'ACL' => 'public-read')

[helios-ag]

@fousheezy, can you provide working configuration for S3?

[bonswouar]

That would be much appreciated !
And the version of the SDK also, if it matters.

[bonswouar]

I searched a bit more, and I really doubt that the current dev-master works on AWS ?!

• First, the $settings['path'] instead of $settings['tmpPath'] in https://github.com/helios-ag/FMElfinderBundle/blob/master/Configuration/ElFinderConfigurationReader.php#L263 hasn't been fixed yet, no ?
• Second, with AWS SDK 3 it's asking for version and region parameters in my case.
• Finally, after I fixed those two issues, I have the 404 Not Found error. I looked into it, and it's apparently because the _stat() method is called on a directory, which is not an object if S3 if I'm not mistaken, so it throws a 404.
  I was trying to do a little workaround, but it's not as easy as it seems, because I don't know well the AWS API (how to identify which ones are directories ? I could use the Size parameter which is at 0, but that's not very clean) and FMElFinder (after I did my workaround... ElFinder wasn't displaying any folder/file at all, and I have no idea why).

[foush]

@helios-ag here's the working configuration I'm using with the master branch:

app/config/config.yml

fm_elfinder:
     instances:
         default:
             path_prefix: "//s3.amazonaws.com/%amazon.s3.bucket%/cmf/"
             locale: "%locale%"
             editor: ckeditor
             connector:
                 roots:
                     media:
                         driver: S3
                         s3_settings:
                             access_key: %amazon.s3.key%
                             secret_key: %amazon.s3.secret%
                             bucket: %amazon.s3.bucket%
                             tmp_path: /tmp
                         path: /cmf/upload
                         upload_allow: ['all']
                         upload_max_size: 5M

[helios-ag]

Hi @fousheezy, which version of aws sdk do you use?

[foush]

presently 2.8.12

My composer.json entry is
"aws/aws-sdk-php": "2.*"

[bonswouar]

Thank you ! I just tried with this version of the SDK.

Though, still have the same problems.. @fousheezy, which versions of FMElfinderBundle and ElFinderPHP are you using ? Without any change ?! Don't understand how it is possible.

First, could you ( @helios-ag ) tag helios-ag/ElFinderPHP with @fousheezy PR please ?

And the Path undefined hasn't been resolve, should I do a PR ? Just need to change tmpPath to path on FMElfinderBundle/Configuration/ElFinderConfigurationReader.php#L263.
EDIT : My bad, I didn't notice there was a path AND a tmp_path.

And finally, I still have the issue :

The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.

You can see here it's because I'm using a "new" region, which supports only V4 signatures.
So to fix that just need to add 'signature' => 'v4' and the region to S3Client::factory().

But finally it goes back to my last error :

404 Not Found (Request-ID: 5450A5CD19727E4F)
500 Internal Server Error - NoSuchBucketException

Because of the S3Client->headObject(array('Bucket' => 'mybucket', 'Key' => 'upload')) (upload is my path, which is a directory in the S3 bucket)

[helios-ag]

@bonswouar, tagged pre PR version with 2.2.2, and latest as 2.3

[bonswouar]

Thanks @helios-ag .

I finally made it work, but I still don't understand how it could work for @fousheezy .

Lots of things modified in the S3 php connector.
I had to change the way of retrieving files & folders (there was a mistake in the preg_match for files, and needed to add another one for directories).
Also I had to add a little hack on _abspath() because of some weird /// in the paths.
Another little hack for _stat() on folders, had to remove the / at the end where there was one.
Finally I replaced Content-Type with ContentType in _stat to properly retrieve metadatas.

I also changed few things : region and signature needed in the bundle S3 configuration.

At the end it seems to work pretty fine, except a little bug on the root folder, which shows no file (except in the tree). I'll take a look at that later.

Should I make a PR ?

[helios-ag]

PR would be nice, especially if you resolve problem with root folder.

[bonswouar]

Ok I just fixed that and made a PR (helios-ag/ElFinderPHP#21)

[bonswouar]

Thanks @helios-ag !