ftp/php file permission conflict: resize doesn't work (NOT A BUG)

[Miika]

This applies to some server setups where php scripts are run under its own user account, for example "www-data" and group "www-data".

a "ls -l" of the UPLOAD folder


-rw------- 1 www-data www-data 3824712 Oct 21 02:34 IMG_4480.JPG
-rw------- 1 www-data www-data 2670216 Oct 21 02:34 IMG_4649.JPG
drwxrwxrwx 2 myaccount myaccount 4096 Oct 21 04:23 ee_temp_upload
-rw-r--r-- 1 myaccount myaccount 421 May 27 16:02 index.php
drwxrwxrwx 2 myaccount myaccount 4096 Oct 21 04:22 test


When ftp user ''myaccount' tries to copy the jpg-files to folder 'ee_temp_upload', it can't because it doesn't have permission to read the files.

I guess php and ftp could be programmed to give uploaded files read+write permissions to group and others as well...

This problem has been there since 1.5 RC 4, I finally figured out why it never worked. All Imagemagick previews and ftp-stuff otherwise work fine.

Regards,

Miika

[Pekka]

This is purely ftp server configuration problem. EE should connect to ftp with same credentials as you would normally connect with, so user groups etc. are same (they are defined inftp server's virtual server configuration, or globally for all clients). If ftp server is configured to treat requests coming from localhost (usually IP 127.0.0.1) differently then you might have above problems. It could be avoided by using real servername instead of "localhost", maybe.

[Miika]

The file server environment is complex - the www-server (apache) and ftp-server are located on different computers, even mysql runs on a different machine.

The problem is that php process runs under its own user account ("www-data") - all files it creates are owned by it, and if no read/write permissions are set, even I can't view the files even they are created in a directory owned by me. (Fortunately, I can manually delete them if necessary)

For ftp login I use my account settings, and again, if no read/write permissions are given to newly created files, the resize function that runs under php user account ("www-data") can not access the files.

I would suggest adding chmod whenever php or ftp is used to copy a file.

---

Two other minor problems:

Install check: ftp test tries to access localhost and 127.0.0.1 - these are not listed on the FTP server setup.

Backup backup/index.php: it should use the global sql server address, not localhost.

Miika

[DavidW]

As Pekka indicates, you're asking EE to cope with a quirk of your environment, and it's arguable that the better solution is to adjust the environment to suit EE if you can. However, it may be that the only way around this is to patch EE; I'm trying to understand what would be best.


Blindly issuing CHMOD commands doesn't necessarily make sense - CHMOD tends to mean little under Windows (EE is not just run on UN*X; Pekka develops EE on Windows, which is why the distribution archive has ^M characters at the end of lines and is a .zip, not a .tar.bz2 or similar).

Even if using CHMOD is an answer, exactly which permissions to grant depends on your environment. Both my local servers and the ones at my hosting provider typically rely on 'other' permissions for Apache and PHP. Apache and PHP run as user www, group www, whilst users have their own uid and are members of the group users, so files are typically owned by myaccount:users. When Pekka suggests CHMODding a folder to 777, 707 is what's needed in my case. Other users on the cluster are members of group users, so they can't exploit the other permission to write to the folder. (In fact, my root directory is 705, so other ordinary users on the box can't descend into my directory structure at all).


I set up a development environment this week to allow me to do some EE debugging; it lives in a ezjail on my FreeBSD box. The current setup is Apache 2.2.3, PHP 5.1.6, MySQL 5.0.24a (I'm waiting for the FreeBSD port to be upgraded to 5.0.26) and ProFTP 1.3.0. All the applications are in the same jail (at least for now), though, as with all jails, they're chrooted to the jail. All the software is installed via FreeBSD ports to allow easy updating; I'm working on upgrading the php-dbg port to the latest release of DBG to help with PHP debugging. I'll contribute the updated port back to FreeBSD via a PR when I'm done.

The 'flavour' script that I wrote to set up new ezjails takes time to ensure that the uids and gids of the users and groups I have in the jail align between the host machine and the jail. This makes life so much easier, not least as I can su to root on the host machine and manipulate file ownership anywhere on the entire filesystem, including in the jails.

For simplicity and for conservation of IP address space, I put all the daemons in a single jail. The jail is in a subnet that is not NATted, which saves overloading my router's NAT table when there's lots of incoming traffic, though it means using a public IP address, which is a precious resource. I could use nullfs to mount folders on the jail running Apache in a separate jail running the FTP daemon (nullfs works really well read only in FreeBSD 6; I'm not sure how well it works read/write as I don't use it that way).

There seemed little point putting the FTP daemon in a separate jail, as the main security benefit of isolating the FTP daemon is nullified by having to give it read/write access to the files relating to the web site. A root exploit on the FTP daemon will allow defacing web sites whether or not the FTP daemon is in a separate jail. I could chroot FTP users more tightly by the use of nullfs and a separate jail, but the security gain is still minimal (unless I'm going to mount every FTP user's home folder separately in the FTP jail using nullfs, which may introduce other issues).

MySQL could be in a separate jail; you can access it perfectly well over TCP/IP anyway (assuming that your database permissions allow for access from the appropriate remote host). There is something of a security gain in making MySQL separate, but it runs under a separate uid and gid to other daemons, which means you'd need to pull off a root exploit to get at the MySQL database files. To stop an attack on MySQL itself, the only ways to the MySQL port from outside my network are to tunnel over SSH (which, on my setup, means you need to authenticate to the SSH daemon using public key) or to use an IPsec VPN (which, on my setup, means you need a certificate on my IPsec CA).

I may move MySQL to its own jail at a later date; it would be as simple as setting up another ezjail, installing MySQL in that jail and copying across the data. There may be some adjustment of user permissions needed, and I'd also have to update the DNS to reflect the different IP address that MySQL would be accessible on.

I refer to the MySQL server as mysql.<domain>, and the FTP server as ftp.<domain>, whilst the web stuff is all virtual hosts. This way, if I do move MySQL or the FTP daemon to other jails or other devices, and hence to different IP addresses, it's just a DNS change.


Back to the main point of what I was saying, rather than the slightly off-topic explanation of the environment I've set up. Hopefully you're using an FTP daemon where you can set the uid and gid of the user based on the login credentials. ProFTP is free and works that way; you can even store the user and quota information in a MySQL database, which helps with reconfiguring things 'on the fly'.

My hosting provider uses NcFTPd, which has, if anything, an even more robust virtual server setup - though it's not free.


In your environment, I wonder if there's any latitude to chown (or get a super-user to chown) the appropriate directories and files as myaccount:www-data, and rely on the group permissions to give access to PHP, configuring the FTP account used for EE to appropriate uid and gid values. It may also be possible to use ACLs if the OS and filesystem you're using supports them, though this almost invariably rules out NFS mounted filesystems.

It would be helpful if you spelled out what user ownership, group ownership and permissions Apache, PHP and external software such as ImageMagick rely on in your current setup (or are you just using GD?), and what your current FTP daemon allows you to do with your FTP user.

The other way around this would be to make a local patch to the FTP code in EE - but I'd prefer to reconfigure the FTP server first.


As for your other two comments, I take the Install Check errors to be entirely benign (though I do wonder why those checks are there myself; they seem to serve little purpose) and the instructions for backup always said to configure the server name in preferences.php.

In the end, of course, what is done to the EE code is down to Pekka.



David

[Miika]

I must apologize, I was totally wrong.

After all it was a wrong 'Server tools' setting.

I didn't notice that resize preview gave error code 1. And the reason to it was wrong setting for "Select Imagemagick's profile removal command type".

This issue is now solved, not a bug.

Miika

[Pekka]

I have seen the IM error 1 in one previous case where compositing watermark image of certain type (unknown specifics yet) gives error 1 although the process works fine. It could be just some notice about some minor issue in composite which is passed to exec as return value 1 (which means it would be IM bug).

One easy solution would be that I change EE return value setting to an array which is added to basecode/SETUP_settings.php. But using multiple approved return values has also a danger that the return 1 is really an error and if it is a"good list" then you would not get error notice in upload system.

[DavidW]

I didn't notice that resize preview gave error code 1. And the reason to it was wrong setting for "Select Imagemagick's profile removal command type".
I think this is where this enhancement I suggested (http://photography-on-the.net/forum/showthread.php?t=229248) comes in. Instead of messing around with the hassle of escaping a * as a +profile parameter, recent versions of ImageMagick support the -strip option to lop off the profiles and metadata. Hopefully Pekka can incorporate this enhancement into 2.02 and make -strip the default for ImageMagick when the EE 2 installer comes out - it should save this sort of problem.



David