Identify where and how background process are started and where they comes from

by ReynierPM   Last Updated March 27, 2015 13:00 PM

First the short story: I need to migrate a server (applications, configurations and so on) and I have no clue about what is in there, no docs, people at charge just abandon and didn't leave any information so it's a kind of black box or black hole. My task, move what is in that server to a new instance and know how things in there works. The problem there are some background process running (see ps -ax output below):

ps -ax

Warning: bad ps syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html
  PID TTY      STAT   TIME COMMAND
  ...
  841 ?        Ss    13:42 python /usr/local/bin/pdoInstaller/
  848 ?        Ss     0:04 php /usr/local/bin/pdoneVendorBroker/vendorBroker.php
  950 ?        Ssl   13:00 /usr/bin/mongod --config /etc/mongodb.conf
  013 ?        S      0:00 CRON
 1014 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
 1015 ?        Ssl    1:02 /usr/sbin/mysqld
 1016 ?        S      0:02 /usr/bin/php rss_article_loader.php
 1065 ?        Ssl    0:29 /usr/sbin/nova-agent -q -p /var/run/nova-agent.pid -o /var/log/nova-agent.log -l info /usr/share/nova-agent/nova-agent.py
 1219 ?        S      0:01 /usr/lib/erlang/erts-5.10.3/bin/epmd -daemon
 1222 ?        S      0:00 CRON
 1223 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
 1224 ?        S      0:01 /usr/bin/php rss_article_loader.php
 1506 ?        S      0:00 /bin/sh /usr/sbin/rabbitmq-server
 1517 ?        Sl    15:59 /usr/lib/erlang/erts-5.10.3/bin/beam.smp -W w -K true -A30 -P 1048576 -- -root /usr/lib/erlang -progname erl -- -home /var/lib/rabbitmq -- -noshell -noinput -sname [email protected] -boot /v
 1728 ?        S      0:00 CRON
 1729 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
 1730 ?        S      0:00 /usr/bin/php rss_article_loader.php
 3137 ?        Ss     0:04 php /usr/local/bin/shareEventHandler/shareEventHandler.php
 3165 ?        Ss     0:04 php /usr/local/bin/repToolBroker/repToolBroker.php
 3180 ?        Ss     0:04 php /usr/local/bin/pdoneLoginProctor/loginProctor.php
 3201 ?        Ss     0:04 php /usr/local/bin/messageBroker/messageBroker.php
 3230 ?        Ss     0:04 php /usr/local/bin/emailBroker/emailBroker.php
 3250 ?        Ss     0:04 php /usr/local/bin/edetailBroker/edetailBroker.php
 3270 ?        Ss     0:04 php /usr/local/bin/cmeBroker/cmeBroker.php
 3921 ?        S      0:00 CRON
 3922 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
 3923 ?        S      0:03 /usr/bin/php rss_article_loader.php
 4395 ?        S      0:00 CRON
 4396 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
 4397 ?        S      0:02 /usr/bin/php rss_article_loader.php
 4498 ?        S      0:00 CRON
 4499 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
 4500 ?        S      0:01 /usr/bin/php rss_article_loader.php
 5781 ?        S      0:00 CRON
 5782 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
 5783 ?        S      0:04 /usr/bin/php rss_article_loader.php
 7242 ?        S      0:00 CRON
 7243 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
 7244 ?        S      0:03 /usr/bin/php rss_article_loader.php
 7575 ?        S      0:00 CRON
 7576 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
 7577 ?        S      0:02 /usr/bin/php rss_article_loader.php
 7705 ?        S      0:00 CRON
 7706 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
 7707 ?        S      0:01 /usr/bin/php rss_article_loader.php
 9368 ?        S      0:00 CRON
 9369 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
 9370 ?        S      0:04 /usr/bin/php rss_article_loader.php
10450 ?        S      0:00 CRON
10451 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
10452 ?        S      0:03 /usr/bin/php rss_article_loader.php
10771 ?        S      0:00 CRON
10772 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
10773 ?        S      0:02 /usr/bin/php rss_article_loader.php
10884 ?        S      0:00 CRON
10885 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
10886 ?        S      0:01 /usr/bin/php rss_article_loader.php
12947 ?        S      0:00 CRON
12949 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
12951 ?        S      0:04 /usr/bin/php rss_article_loader.php
13573 ?        S      0:00 CRON
13574 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
13575 ?        S      0:03 /usr/bin/php rss_article_loader.php
13963 ?        S      0:00 CRON
13964 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
13965 ?        S      0:01 /usr/bin/php rss_article_loader.php
14157 ?        S      0:00 CRON
14158 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
14159 ?        S      0:00 /usr/bin/php rss_article_loader.php
16083 ?        S      0:00 CRON
16084 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
16085 ?        S      0:04 /usr/bin/php rss_article_loader.php
17089 ?        S      0:00 CRON
17090 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
17091 ?        S      0:03 /usr/bin/php rss_article_loader.php
17103 ?        S      0:00 CRON
17104 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
17105 ?        S      0:01 /usr/bin/php rss_article_loader.php
17553 ?        S      0:00 CRON
17554 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
17555 ?        S      0:00 /usr/bin/php rss_article_loader.php
19227 ?        S      0:00 CRON
19228 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
19229 ?        S      0:04 /usr/bin/php rss_article_loader.php
20318 ?        S      0:00 CRON
20319 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
20320 ?        S      0:01 /usr/bin/php rss_article_loader.php
20375 ?        S      0:00 CRON
20376 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
20377 ?        S      0:02 /usr/bin/php rss_article_loader.php
20722 ?        S      0:00 CRON
20723 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
20724 ?        S      0:00 /usr/bin/php rss_article_loader.php
22324 ?        S      0:00 CRON
22325 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
22326 ?        S      0:03 /usr/bin/php rss_article_loader.php
23549 ?        S      0:00 CRON
23550 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
23551 ?        S      0:01 /usr/bin/php rss_article_loader.php
23643 ?        S      0:00 CRON
23644 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
23645 ?        S      0:02 /usr/bin/php rss_article_loader.php
23945 ?        S      0:00 CRON
23946 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
23947 ?        S      0:00 /usr/bin/php rss_article_loader.php
25875 ?        S      0:00 CRON
25876 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
25877 ?        S      0:03 /usr/bin/php rss_article_loader.php
26840 ?        S      0:00 CRON
26841 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
26842 ?        S      0:02 /usr/bin/php rss_article_loader.php
27223 ?        S      0:00 CRON
27225 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
27227 ?        S      0:01 /usr/bin/php rss_article_loader.php
27538 ?        S      0:00 CRON
27539 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
27540 ?        S      0:00 /usr/bin/php rss_article_loader.php
29374 ?        S      0:00 CRON
29375 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
29376 ?        S      0:03 /usr/bin/php rss_article_loader.php
30232 ?        S      0:00 CRON
30233 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
30234 ?        S      0:02 /usr/bin/php rss_article_loader.php
30444 ?        S      0:00 CRON
30445 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
30446 ?        S      0:01 /usr/bin/php rss_article_loader.php
30682 ?        S      0:00 /usr/sbin/apache2 -k start
30683 ?        S      0:00 /usr/sbin/apache2 -k start
30848 ?        S      0:00 CRON
30849 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
30850 ?        S      0:00 /usr/bin/php rss_article_loader.php
32692 ?        S      0:00 CRON
32693 ?        Ss     0:00 /bin/sh -c cd /var/www/pdone/ipad_v2/rpc; /usr/bin/php rss_article_loader.php
32694 ?        S      0:03 /usr/bin/php rss_article_loader.php

Some are running through CRON I have that ones identified, the easy path, already but others like the ones started by PHP meaning for example:

 3137 ?        Ss     0:04 php /usr/local/bin/shareEventHandler/shareEventHandler.php
 3165 ?        Ss     0:04 php /usr/local/bin/repToolBroker/repToolBroker.php
 3180 ?        Ss     0:04 php /usr/local/bin/pdoneLoginProctor/loginProctor.php

I can't get where that comes from and I need to know where and how they are started in order to setup the same on the new server, can any gives some ideas in how to attack this problem? The only thing I know at this moment is RabbitMQ use that script to deliver messages and do some tasks. I need to identify where and how background process are started and where they comes from. Original server is Ubuntu the new one is CentOS is not a problem but just FYI, any can give me some help or ideas?



Answers 2


Start by investigating the relations between parent/child processes using a command such as ps -AF --forest. The Parent Process ID (PPID) will either be the process that spawned the process in question, or 1 if it forked or became orphaned. The --forest switch to PS displays a graphical representation of this relationship.

Deamons are usually started with scripts in /etc/rc5.d/, try looking there. Also look at the files for cron tasks. The locations and setup for these differs depending on Linux distribution, consult the distribution documentation for details. For the upstart init system, it's possible to list all daemons using the service --status-all command.

If those comes up empty, you can try searching the filesystem for text strings that match the commands used to start the processes. For example grep "rss_article_loader\\.php" -r /usr /etc will show all files in /usr and /etc containing the string 'rss_article_loader.php'. Note that grep searches with regular expressions and not simple text strings.

Julian Sivertsen
Julian Sivertsen
March 27, 2015 19:50 PM

I had luck identifying culprits with the Linux auditing subsystem.

Linux audit files to see who made changes to a file

linux - Monitoring system calls - Information Security Stack Exchange

ivan_pozdeev
ivan_pozdeev
March 27, 2015 21:49 PM

Related Questions





Edit a command send to background

Updated April 06, 2017 13:00 PM