LinaeAzn

Joined

1,010 Experience
2 Lessons Completed
1 Question Solved

Activity

Posted in How do I access files on server with a cron or similar ? (outside of app)

I got an answer from Chris on the cron/whenever gem :
 Cron jobs with Whenever run in a special shell setup and actually load ~/.bash_profile if it exists. So you actually need that configured to load rbenv so your jobs run. 
Apparently, Chris fixes that hatch config on my server modifying my deploy user bash_profile. My server was installed/provisioned before he got this fix for hatch, so he had to do it manually for my VPS specificaly.

I can't test right now, but i'll update when I can confirm that all works well.

Posted in How do I access files on server with a cron or similar ? (outside of app)

There's nothing related to database when I execute "env"  from the linux console.

I looked into database.yml, there's a "url" var :
production:
  adapter: postgresql
  url: <%= ENV['DATABASE_URL'] %>
  encoding: utf8
In hatch, the ENV['DATABASE_URL'] looks like : "postgres://my_randomized_database_username:my_database_password@127.0.0.1/my_app"

Do I have to export this full url to my crontab ?
Yes, yesterday, I did send an email to Chris from hatch support for that and the symlinked folder between releases !

Posted in How do I access files on server with a cron or similar ? (outside of app)

For the bit change, I failed to understand one thing. My files was "-rwxrwxrwx" (so 0777 if i'm not wrong) when i got permission denied. I switch to 2777 then back to 0777 without changing anything. And my files were accessible/changeable again.

This is my crontab, and yes the RAILS_ENV is there :
# Begin Whenever generated tasks for: my_app at: 2018-02-27 00:49:52 +0100

PATH=/home/deploy/my_app/shared/bundle/ruby/2.3.0/bin:/home/deploy/.rbenv/versions/2.3.3/bin:/home/deploy/.rbenv/libexec:/home/deploy/.rbenv/plugins/ruby-build/bin:/home/deploy/.rbenv/plugins/rbenv-vars/bin:/home/deploy/.rbenv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games



20 01 * * * /bin/bash -l -c 'cd /home/deploy/my_app/releases/20180227004925 && RAILS_ENV=production bundle exec rake bien --silent'
Got the same error message with : "PG::ConnectionBad: FATAL:  role "deploy" does not exist"

And YES (finally), calling the rake task directly from console works with : 
bundle exec rake bien RAILS_ENV=production

Posted in How do I access files on server with a cron or similar ? (outside of app)

I make baby steps.
I changed my call to a rake task as you told me.

Then ran into a really WEIRD issue (at least for me) :
deploy@vps:/home/my_ftp_user/ftp/files$ mv my_xml.xml my_xml.xml.old
mv: cannot move 'my_xml.xml' to 'my_xml.xml.old': Permission denied
deploy@vps:/home/my_ftp_user/ftp/files$ ls -l
total 38632
-rwxrwxrwx 1 deploy            deploy               87084 Feb 24 15:10 my_xml.xml
-rwxrwxrwx 1 deploy            deploy            31853133 Feb 26 23:12 my_zip.zip
-rwxrwxrwx 1 my_ftp_user       my_ftp_user        7614269 Feb 26 23:17 my_second_zip.zip
I can't mv/cp/rm a 777 file that got the right owner...
=> After that, I chmod 2777, then 0777 back, and got it working. I really, don't understand what just happened. (rights et owner are exactly the same now, but deploy user now have the permissions to do things on the files). So it seems to be fixed, wait and see on that one I guess.

The cron now, seems to start, but run into :
rake aborted!
ActiveRecord::NoDatabaseError: FATAL:  role "deploy" does not exist
My database is auto generated by hatch deployment, and got a username randomized, and it's not "deploy".

Tomorrow I'll try to search if there's an easy fix for this one.

For the all thing, I have the feeling that I do/did  some newbish errors that it make this not as smooth as it should be ! Thanks again for the help and your time Jacob.

Posted in How do I access files on server with a cron or similar ? (outside of app)

Yes 777 was just to test permissions on files, without success !

I already did add the user deploy to the group of my restricted FTP user.

Here's the crontab -e
# Begin Whenever generated tasks for: my_app at: 2018-02-25 19:04:12 +0100
PATH=/home/deploy/my_app/shared/bundle/ruby/2.3.0/bin:/home/deploy/.rbenv/versions/2.3.3/bin:/home/deploy/.rbenv/libexec:/home/deploy/.rbenv/plugins/ruby-build/bin:/home/deploy/.rbenv/plugins/rbenv-vars/bin:/home/deploy/.rbenv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games

25 18 * * * /bin/bash -l -c 'cd /home/deploy/my_app/releases/20180225190348 && bundle exec bin/rails runner -e production '\''Bien.cron'\'''

# End Whenever generated tasks for: my_app at: 2018-02-25 19:04:12 +0100
I had to include "env :PATH" because of previous errors of cron not finding rails/bundler

I do not use a rake task, but a runner calling to Model.method
And my schedule.rb looks like this : 
env :PATH, ENV['PATH']

every 1.day, at: '06:25 pm' do
  runner "Bien.cron"
end

And to explain what my method do :
- clean specific files (xml) from folder
- unzip a zip file, containing xml + img
- read xml, geolocate some addresses, insert database

Posted in How do I access files on server with a cron or similar ? (outside of app)

Hi, 
I finally took the time to makes all the changes this weekend. 
I've watched the video on the whenever gem, and managed to make my cron "working".
But i ran into issues :

With your advices I moved back the homedir of the user outside of my app and created the symlink.

1. As I said before, I use hatch from Chris to deploy my app, but the symlink is not "following" the current release.
With : 
ln -s /home/my_restricted_user/ftp/files/ /home/deploy/my_app/current/cron/
Where "current" is already a symlink to the last release of the app.
Do you know how can I make this symlink persistent with each deploy ?

2. Even when I got the symlink right with the actual release, my cron had trouble executing. With this error :
bundler: failed to load command: bin/rails (bin/rails)
Errno::EACCES: Permission denied @ unlink_internal - /home/deploy/my_app/releases/20180225184145/cron/files/my_xml_files.xml
At first I thought this was some permissions issues. So I changed the owner/group of file to deploy:deploy (same issue), then I put chmod 777 on my files (again same error message).

After that, I tried to just delete the symlink, copy the directory, and put 777 on files. And know, it is working.
Do you know how to make those symlinked files accessible from the rails app / cron ?

Thank you for all the previous answer Jacob, it helped me learned some new things !

Posted in How do I access files on server with a cron or similar ? (outside of app)

Thanks for the answer, 
I'm not really good with servers, but I achieved to put homedir of the restricted FTP user in my deployed app.

I managed to get my files outside of my public directory (root of my app) because of what you said on security.
Then I change the paths of file with : 
Rails.root + 'cron/files/...'
This seems to work on url call, which is a good point.
Now I have to install the "whenever" gem to get this cron running for the first time and delete my route to remove access by url.

Do not hesitate to tell me if I'm doing some bad rails design/workflow.

Thank you Jacob for pointing me in the right direction !

Posted in How do I access files on server with a cron or similar ? (outside of app)

Hi everyone, i'm a bit new to rails and i'm searching for help.

I have a rails app where I receive a zipfile on my server (outside of the app, homedir of new FTP user).
Then I want to unzip the files (xml and img) and do my things to update my database with the xml file and moves images to the right directories.

I already coded all of that, and this works perfectly, but today I have to put the zipfiles on the /public directory. To make this work, I created a route to a controller:action. I just have to call my url.

Now, I want to make this automated with cron.
Problem is : my files can't be in public directory of my app (ftp user for zip/xml don't have rights to put it there)
And I have no idea where to start and how I can do that. Anyone can help me with this ?

PS : I use hatchbox to deploy my app.

Here's a part of my code :

def unzip_files_cron
  require 'zip'
    
  FileUtils.rm_rf(Dir.glob('public/cron_images/*'))
  File.delete('public/cron/annonces.xml') if File.exist?('public/cron/annonces.xml')

  Zip::File.open('public/cron/annonces.zip') do |zipfile|
    zipfile.each do |f|
      if f.name == 'annonces.xml'
        f_path=File.join("public/cron/", f.name)
      else
        f_path=File.join("public/cron_images/", f.name)
      end
      FileUtils.mkdir_p(File.dirname(f_path))
      zipfile.extract(f, f_path) unless File.exist?(f_path)
    end
  end
    
  File.delete('public/cron/annonces.zip') if File.exist?('public/cron/annonces.zip')
end