let's see oh let me turn my camera on a little bit so I can see
myself there we go
okay let's see oh I need to open the chat there we go
excellent Martin is here Sean and Josh himself
smoked out cave in Canada nice I need a hat I need a hat today let's
wear a hat there we go now I feel better okay Mr Francis
excellent Mr Workman is in the house the other Mr Workman the better Mr Workman
cool Mr Williams Marcelo thank you for coming on Dr Bob
excellent clarinet hey everyone gotta check out uh shoot I'm drawing a blank now that I saw
clarinet I know whose website it is he posted a nice little page on the community was uh using uh cookie jar
with rain check in the new playstack it's a nice one
excellent Dave here Mari thanks for popping in I appreciate it everyone go
check out mari's YouTube channel I think she just dropped a few videos recently at least I just watched some recent ones
good stuff she redid her husband's website she kind of does a whole like breakdown of that so check out Mari
Pfeiffer's uh YouTube channel check it out um
see Mari I'm watching you you didn't know I watched that did you okay so
um let's just get into it as you probably know Monday
um early at least early Monday morning my time of course I start a live stream and someone rings the doorbell
um but early Monday morning I uh got hacked
um so I got a call hopefully my dogs aren't uh annoying you guys too much
um so yeah early Monday morning I got hacked and uh yeah it was uh it was a long Monday
um and uh but luckily we got it all fixed and uh and we're good to go now I
thought it would be fun I I did write up an entire post on the community if you haven't read that it's uh I've gotten a
really great positive feedback on that post um so go check that out but we are gonna be diving into it today and it might
help with some visuals right
so how did it happen or let's just say what happened first um I figured out that uh this particular
person uh exploited a hole in my total CMS demo
okay maybe I should share my screen right now we'll just kind of talk about uh what happened and I'll show you okay
uh so let's go ahead and uh give me one second
right so um
if you go to the total main total CMS website and you go to the demo okay in
this particular instance the soccer demo uh was the one uh was the demo that was actually exploited okay
um now because this is total CMS and it is a demo I want I want to give users
the ability to edit the demos they can play around right so you can go to edit this page
and then on this particular page you can actually edit the contents of the demo
now I do have jobs that basically run every 30 minutes to reset everything back to how it should have been okay
um but the getting right down to it the problem was this little area right here
okay and what this particular user figured out
was that if he were to upload a PHP file into this Depot this is a total CMS
Depot essentially you can upload previous to an update that I already shipped on
Monday you could upload whatever file you wanted right so I can go ahead and
let's just uh you know I can have an image I can go ahead and upload this image now that is jpeg it is allowed
right however if I were to go into here and upload let's say a PHP file okay now
this fails previously it did not um so
he had uploaded he figured out that he can upload a PHP file and once that PHP file is on my server
um it is a predictable path in terms of where that file is located
so essentially he um was able to upload a PHP file and I actually will show you the PHP file that
did the most damage um I have it on my local machine and I will show you exactly what it is okay
um so once he he knew this PHP file was uploaded onto my server basically he
just went to the predictable path of that PHP file right if you didn't know
inside Depot you can go ahead and there is the path to the files okay and uh
basically you can just put that in there and uh use that use that PHP file and do
whatever you like now let me preface this before we go further that
by default right you shouldn't have your P your admin areas for total CMS 100
publicly accessible right had my admin page be locked
then none of this would have happened at all right um if it had been password protected he
wouldn't have been able to get to this page to be able to upload a file okay um so luckily hopefully none of you have
you know your admin Pages not password protected right so uh rule number one
password protect your admin Pages um either with the total CMS protect stack or with page safe or with sitelock
or with whatever else you want to use okay
[Music] um so let's look at what this person
um uploaded to my server
so he he had these are copies now uh luckily total CMS does auto backup so I
have copies of every file this this user uploaded um most of these were
um didn't really do much okay or he couldn't because of how my server was
configured and whatnot so um I'll kind of go through some of the things
um that he attempted to do um but I think that I'm gonna spend most of the time on this guy right here which
was which was pretty much as you'll see I'm going to launch let me go ahead and um
I'm going to launch it just a really quick a local PHP server on my Mac I'm running only this on my Mac so that uh
um we can just go ahead and do that I'm gonna be doing probably a lot of really techy stuff throughout this to kind of
show you um I'm not going to teach so um I'm just going to kind of show you
what I did um I may tell you about some other videos that I did that kind of talks about some things in depth but um all
right so as you saw this is the page that was uploaded to every almost every
web page every website that's on my server that's on This Server I do have multiples or web servers but I recite
that was on this particular server um had this as the home page okay
um so what exactly did he do how did he do it so um as I said we had this file called
he called it zero.php okay and this is what it was right it was a file browser
so he uploaded this to the server it allowed you to see all the files on the file system
um you can go up into let's say downloads right yeah I mean again this is all my local Mac okay but you kind of
get the drift you can see you can go ahead and
um click on a file and you can see its contents okay you can
um browse and upload a file okay uh just to make it a little bit
easier for them to upload files rather than having to use Depot as a hack okay he was able to upload files directly
from this now because all of my um websites are on the same server he
was able to basically use this to kind of move and upload files around on my
server okay so um hence he replaced the home page on every web page every website
um so this was the file that did the most damage okay um so there we go pretty crazy right
yes I'm using the Posh name of person yes um yes
um so yeah pretty crazy um
now what was what were the fixes um so first off
um as you well as you saw oh shoot I close that browser window didn't I oh well um when I tried to upload
um a PHP file uh I have now blocked that if you're running the latest version of total CMS
um it will not allow you to upload PHP files as well as some other file extensions that were
questionable that we should allow right so um basically any sort of known script or
executable thing that could be ran on the web server um has been denied with Depot
okay um maybe in the future we can allow you to allow certain types if you
if anyone really needs to but I can't see a need for having to upload a PHP file via Depot
um just seems like a just a bad idea okay um as we as we saw okay so that was
definitely a good quick fix um you know for total CMS that we can uh
basically block allowing upload of any file that you want okay um so basically all script or anything
that's dangerous to have on your server um is not allowed to be uploaded okay
but all your common types are still work probably 99.9 of the files you are going
to want to upload is just fine zip files now if you wanted to upload a PHP file that could be used
you could zip it and upload it I guess but I'm at that point it's just a zip file okay
um so it really it does a lot less damage okay um at least to your web server
um execute you know windows executables dlls in Windows land there's a lot of
different ones I did a little bit of research um there's a lot I'm actually going to be shipping a small update today that
has a little bit more breadth of especially for Windows servers with that said don't run a Windows Server just run
a Linux server please like it'll make your life so much nicer okay but there's
a lot more stuff I did a little bit more research on the Windows server side for the few people that don't have a choice
um that I'll ship an update for that has a little bit more um you know blocking Logic for some
Windows Server stuff so there we go but with that said if you have a choice run Linux make your life a lot easier
um okay but I know not everyone can uh can do that
uh what does the person get from hacking my website um
he gets to be known as a big MF basically I mean he he got nothing other
than just being malicious uh there there was no point he didn't he didn't gain any data he just played a prank it's
like I guess the website version of a doorbell ditch or toilet paper in someone's house you just get some
fun out of it that's the only thing he didn't get anything off my website okay
um so there was a question that asked was any of or any passwords or any like any
of your data compromised no none of your data lives on This Server um and I do have a full track record of
every command and everything he did and I'll show you how I figure that out um in a little bit but um yeah I followed
it followed his entire trail of everything that he did and um yeah
nothing was nothing malicious was done it was all just um it seemed like he was learning
luckily he was he could have done potentially more damage um to me um he wouldn't have got anything from it
other than just being malicious um but basically it looked like he was just learning the ropes of of how to
potentially do something like this um so anyway um because of my server configurations
he wasn't allowed to do some of the things that he did attempt to do um
such as uh it's really a lot of this has to be nothing how what you guys can do like it's you know the the Linux user
the account that the web server is running under and um can that account what does it have
access to privileges to other folders or is it just scoped to the web server folders right which is important that's
what you want um can it run schedule jobs no you don't want it to be able to but
depending on your server setup um things could have got a lot worse so luckily this server was built by me
um and um while I'm probably not the world's best Linux admin I have been doing it
for quite a long time um so I I did know and I had at least
the basic security measures in place okay um
there is another thing I should note that um I did have some I did tweak my PHP
settings um in my PHP ini file um I did disable
um hold on let me see if I can
there um so I I still have it on my clipboard look at that uh where is this let me put this inside
create a new file here
all right here here's a here's a good tip okay I don't think you need print but
all right so inside your if you have control over your PHP ini uh whoops I
made it way too big okay um I think this line would uh would
definitely not hurt you okay um and it basically what it does is it disables the exec and system commands in
PHP which basically allows you to run um command line
jobs via a PHP file um so a lot of hosts have this disabled
um like for example if you're using chili dog chili dog already has these things disabled okay
um now this does mean if you have a PHP you won't be able to run the exec or system commands Okay in your PHP Scripts
so um but that could save you because then that means no one can upload a PHP
file that could potentially run a command okay so um this is a a nice to have again if we
stop the user from ever being able to upload a PHP file that is nice okay but
uh this wouldn't be a horrible thing to add to your server it's just another layer of protection
um if you don't need those two commands Okay there we go all right
um forget where I was going next oh we were talking about what did he get from it
um yeah he didn't get anything from it um except just being a douche and um
yeah making my life hard for a day um you know all in all I'm
trying to make lemonade out of lemons here um you know this did open up my eyes to uh a loophole in
total CMS that loophole is now plugged and that is a good thing right the only good thing can come with that not only
does it help me but it helps all of you guys right yes it's a caveat where this
only this exploit was only done because my demo page is not password protected on purpose
um but it's a loophole that is now plugged so that uh all of you
have a better more secure CMS so that's only a good thing so making lemon
lemonade out of lemons there we go always a good thing
um so there were no viruses or bugs Left Behind no now I will show you how I was
able to discover that and uh
it's because of my server setup that I was able to actually Rectify I had my websites back up and running in less
than an hour okay and everything that he did was reverted in about an hour okay
um at that point then I had to do some investigative work to figure out what the root cause was and that took a lot
longer that took multiple hours to figure out what was going on but I at least had everything back up and running
with all of his files removed in about an hour or less okay from being woken up
at 4am thanks Josh okay so um so yeah there we go
I did not find any malware um now since we're on that topic I will
am I sharing my screen yes I am still sharing my screen okay um
I did a live stream uh four years ago now and I
realized the date because funny enough um I was talking to Josh uh Stax Weaver
Josh um about this a week or two ago about how I manage and deploy my websites
using something called git it's a Version Control System now this is the part um that is going to
be highly technical um and I'm not going to go into the how to's I do that I did that in the live
stream four years ago my workflow is mostly the same now um but
I do use git and I will show you the benefits that I got using git okay to
deploy my websites instead of using SFTP or FTP okay
um and I do apologize this is technical okay
um but I will show you the benefits um so right now I'm on my web server and
I am on I'm on made for Stacks uh website okay and um I'm just going to run a
couple commands uh actually here let me go ahead and I will make the text a
little bit bigger for you guys there it goes just so you can see
a little bit better a little more
all right there we go all right so it's nice and big okay so I'm going to run now this is all terminal based stuff
okay so I know just by having that do this on Terminal it's going to blow some
of your minds okay but um like I said I warned you multiple times now this is technical so I'm going
to run git status and that tells me the status of my git repository this is on my web server
um inside the made for Stacks website and um it it basically says I'm up to
date with my Branch um it did find a couple files that it didn't know about and this particular
one I'm okay with it's just some cash files for feed um for feeds and so I'm okay with that
okay if I wanted to I could put these into my git ignore file and it would ignore these not give me any errors okay
but um what I want to do is I want to show you if I I'm going to just create a file
right now I'm just going to create a file called touch P dot PHP okay and if
we look that file is now there now I I put that file in there that was not
added to the server via git via this uh Version Control System okay
and now I'm going to do a status check again and via the status check it sees git
knows this file that I don't know about is on the server
what's up with that okay another thing that it can do is let's say now this particular hack
didn't modify any of my well it did it modified my homepage it basically replaced my entire homepage right so
let's go ahead and I'm going to I'm going to edit um this robots file really quick okay
just for fun and I'm just going to put in a couple lines
okay so I have modified this robots file whoops
okay I've modified that robots file so if I look at the contents of that robots
file it now has this line that wasn't there before okay now I'm going to check my status
again and if we see now um it says right here look
modified robots.txt is modified and again we have these untracked files okay
this particular one I know about it's just cache files for feeds but this p.php file wow that's not a good one
right and of course if there are multiple modifieds it would show you everything that's modified if there are multiple untracked files it'll show you
all the files it doesn't know about okay now what's really cool is
in order to revert back to whatever I know is the golden standard for this it's just a simple Command right so I
just do git reset hard head okay kind of a funny command yep let's get reset hard
head is the uh where I want to go to and now it's so it's reverted okay and
that what that does is that will un that'll undo all of the changed files
right so if you see robots.txt is no longer um no longer has that uh that
line at the bottom that I added no longer there okay and for this untracked file I just need to delete it so I'm
just going to do RM Dash f P dot PHP
now if I run the stat the status command again voila there we go okay we're down
to this one but again I know about that right so basically what I did is I went
through each of my websites and I just did this um when you know what you're doing this
takes me literally like a minute or two for each uh website and I have about
I think 15 or 20 websites on this server okay um and not all the websites were
um were compromised but um a lot of the rapid over ones were funny enough so
um so there we go that is kind of how I
fixed things on the server okay is all with Git okay pretty cool okay
um not gonna really dive much into that if you want more examples or more in depth about that I do
um you know go over that in the live stream I did four years ago just searched my live streams for um
get publishing with get or Advanced publishing something like that on my YouTube channel
okay um next up so that's how I reverted everything and
I I did the initial fix now you might be wondering how in the world did I figure out
where the the root cause was how did I figure out that this is the
downloads of this page was what was triggering her at all and yes that took me
hours okay um to figure it out now
um again it did it did take me hours and basically what I did was
I searched through so I downloaded all of my access logs
okay from my server and now
uh my particular web server right now isn't running Apache um earlier this year I moved to a new a
new server called caddy um so my logs are going to look quite a lot different than yours
however most of you are probably probably running Apache and they have
access logs as well and um it'll have the same general information essentially
um it logs every single HTTP request that was ever made to your
domain so you can see everything every time anyone visits any URL on your browser
it shows up in these access logs and basically what I did is I kind of
um I went into terminal and just I'm I am a terminal junkie and that allowed me to process these a lot quicker because
obviously I wanted to only find the sketchy um the sketchy requests right
um so let's go ahead so I I'll be honest I I didn't document
everything that I I did in terminal to to figure it out but essentially I I got
around to finding um you know stuff from this soccer Depot okay
um I then filtered out into its own kind of log file and I had um basically every command that this guy
was running if you see here's the zero.php file okay now now if you look what's important here
um if you look this zero.php is not in Depot he basically
moved this in multiple locations across multiple websites and inside multiple
folder like random folders you just like put these files in random folders all over the place okay so that
it would be hard to detect now I use git so it was super easy for me to know
everything that changed I have to admit even when I did my my
live stream I didn't see protecting myself from hackers as a as a
bonus of using git now it saved me probably days of
headaches um of or potentially never knowing what the heck happened okay
um because I was able to find if you look here I have free Stacks browser detective Stacks files xero dot PHP
right he knew he put it there but I I mean I have so there's so many folders and places that potentially could be a
file that I wouldn't know about especially if you're just you just use Rapid even you just hit publish you have
no clue about the structure of your website right um but because I use git git knows so
git knows about everything that um isn't expected to be there
so um yeah with this I was able to find out every file and every request
and then basically I sorted all of these by date uh and when I sorted all of these by dates I could find the very first
instance and that first instance led me to this Depot
uh which then obviously I know total CMS and then it just clicked on what he did I was
like oh he found that loophole right and that's where we are now I plugged in all
the loopholes um and whatnot so what are some other things
um that I did um let's see uh oh I'm seeing some questions here I'm sorry I have total
CMS on a demo server of my server without registration code do I have a security problem or should I just make
the update yeah just just release the update Marcelo um so yeah that update will will at
least plug um so that people can't upload PHP files and whatnot okay
um I do recommend as a as a precaution I have that PHP ini thing that I showed
earlier um you can watch the replay um and uh and get that uh but that that
would be a nice that's not required again that's just an extra little you know security thing okay hopefully your
host is doing that already okay but there we go um so yeah if you have a demo that is live
um it doesn't matter if it's registered or not um it doesn't need to be registered obviously if you're not registered and
the license expired you're not gonna be able to upload stuff anyway so um there's that
um so there we go okay let's see are you self-hosting are using convert I
self-host um so Mac assist I do host I run my own server I self-host well it's it uses
digital ocean so it's not my server I do pay for the server space but I I'm the
one that set up the entire server I did it all myself from scratch okay um so if you if you need
um you know chili dog is a great one um I I recommend them he has he's very security conscious so if you're a very
security conscious Guy Greg over at chili dog is a great job um there's other I do use digital ocean
it's not for the faint of heart though it's very technical if anything goes wrong it's your your
responsibility I couldn't go to digitaloce and help me with this it's my server I did everything okay so I'm on
the hook for it all um another server that I uh a host that I
use for some test websites and some smaller stuff that I play around with I use dreamhost it's really great I have
affiliate links uh to both all of that on Weaver space start if you want to use
those okay [Music] um
hey Mari says so someone Josh notified you that yeah Josh notified basically
um yeah Josh notified me he called me at 4am I was sleeping in bed uh that my sights were down
okay and uh that just prompted me to go to my uh fire up my laptop and again I
was on the hook for it all so um I had to fix it
I doubt my clients will get a call about a hacked site uh
yes so um so Mar you're like well what how does this affect you so
um if you don't manage your own server I'm pretty much I'm sure everybody here
doesn't manage their own server right so you just can't fire up and do everything
that I did okay um well maybe you could uh you know a lot of hosts do have shell access and
and whatnot right um but again I I had a very technical setup so
um but not everybody has that so have had your site been hacked
um hopefully if you're using a traditional host um they're already security conscious since
they host tons of users websites they're gonna have a lot of security measures and points so stuff like this doesn't happen
okay um and again also password protect your admin page had I had a password on my
admin page none of this would have happened anyways okay um but let's say something else happened
and your site got hacked um one thing you could do is obviously
contact your host okay um and then have them revert to a backup
I strongly urge that um if you don't have something like my git setup because
it's going to be very difficult to find any files that are buried in the system
so you can either revert to a backup or another thing that you can do is
delete your published website delete the entire
thing if you have total CMS make sure you get the CMS data folder okay first you back up that CMS data folder
clean the entire site from your server do a republish all and then put the CMS
data folder back what that will do is that will give you a 100 clean website and you can
guarantee that no files are like buried somewhere in the file system right
I'm not sure anymore let's just go to me um so hopefully that makes sense right so you can either revert to a backup
that your host took that has a known good version non-hacked version of the site
then you also need or you can uh clean the server up basically delete
everything that's on the server re-upload a new copy from from rapidweaver a published website
um be careful of that's if you are using total CMS making sure that CMS data because that is not stored inside rapidweaver that is only on your server
so before you trash everything make sure you get that CMS data folder okay
um and then republish all put the CMS data back and then you will be you're certain to have a clean copy of your
website okay with that said you also need to make sure you find the root cause on how the guy got in right and
hopefully um you don't need to go through all the steps that I did hopefully if you're using a host they will be able to
identify that okay so hopefully that that works
would it be possible to restrict Depot to a particular file type such as PDFs yes that's already a possibility
um so you can restrict Depot to only allow PDFs or only allow zip files that's already a feature again it's just
the perfect storm of how I had my demo set up I allowed any file to be uploaded and I had it not password protected
okay
yes Dave hitting says make sure you don't work don't delete any of your Warehouse documents so yeah uh I I said
about CMS data but if you have a warehouse folder make sure you don't trash that too okay and maybe even look
into there in case like if you need those files make sure you look into that there's there's nothing suspicious in
there okay but hopefully you guys have a backup of locally of your warehoused
folder um that's always a good thing to have as well okay
Dave I have backup Synology once a week yes cool
um the CMS folder is where the malicious file will be stored not necessarily because um he with PHP you can create files in
other directories right so um like I showed a little Glimpse earlier he had moved at zero.php file
yes he originally downloaded it to the um Depot folder however uh he then moved it
right so kind of crazy um so he used PHP to basically move that
um to different locations uh within the web folder right so there we go he made
copies
what do you think about the idea to allow a limited number of inputs for page safe a limited number of password
entries um um not sure I understand uh maybe
you're thinking password attempts um so I I mean
with paid safe you you CA you do have multiple options you can have a passcode which is a lengthy long password which
is going to be just as strong as any other password authenticated engine um in terms of blocking a particular
user um after a certain number of failed attempts that's something that could
potentially be implemented I do have that on my list of things to look into I
don't know if it will get implemented but I I have already started page safe too to just uh um but I have zero ETA on
that um I just did some work on it actually made for stacks.com actually is running
page safe too because I needed some cool features for that um so um I did start page safe too but
um yes that is an idea I have that on my list of possibilities I think there could potentially be some better ways
than just that um though so good idea
let's see lots of questions I think I've answered all the questions
as Scott Williams says sometimes it's not new files sometimes they do modify existing files yes so that's why I
showed you earlier how I modified that robots.txt file now that particular file a hacker isn't going to care about
robots TSC file but they can modify JavaScript files they can modify PHP
files so that maybe you don't even notice like um so when the user hacked my site right
uh hopefully I still have that server running
all right so when this particular person um hacked my sites they replaced the
home page with this super easy to know that
I got hacked right however if there could be a lot more subtle and
clever hacks right which is just and and it potentially you can you could have someone hacked your website and not even
know it because your website's still 100 functional but they could modify a JavaScript file
that maybe sends data or injects some stuff on your website that you don't even know is there they can inject some
PHP that does some stuff in the background but still allows all your PHP to run right so not knowing
um is probably uh definitely a possibility where someone is modifying
an existing file on your site still allowing your site to function but
then doing extra stuff underhanded stuff maybe data collecting or all kinds of stuff everyone say hi to my son you're
on my live stream Josh [Laughter] he was waiting for the opportunity to
win to get the papers um
yes as you know oh Jeff Taylor I never upload from rap Weaver I always upload from using transmit I mean uploading
from transmit versus rap versus rap Weaver doesn't really uh
while that's great a transmit is great because it has that sync function okay which is the bee's knees
um that won't necessarily stop a hacker right that and that's not gonna prevent from hacking that's just giving you uh
FTP is still FTP whether or not it's from rack Reaver or Stacks or transmit okay
um it's just the feature of that app um
so always do a very oh but he's saying that basically he always does a recent clean publish so that he can you know
basically then sync that up with transmit yeah that's cool um good job
perfect thanks Jeff yeah you made a good point Yeah so basically you know on a weekly basis he cleans the server and
then syncs it from a local folder using transmit um so yeah that works
um I do a similar thing but I use git uh and get as I showed earlier gives us a
little bit more extra control um and some some cool features because it knows what's changed what's not
changed and all that jazz but uh yeah if you're doing clean installs then you're good to go
cool um well guys uh I I think we're we're good I I've kind of shown on everything
um I'm happy to if you want to continue this you've got more questions and you want to chit chat on the community about
um what happened and um more details and questions about what I did to circumvent it
um let me know um yeah I am pretty happy that this I mean I'm not happy that it happened but
again um lemonade out of lemons here um I have some great updates shipped out for total
CMS already um so we're definitely more rock solid and secure um and uh yeah make sure you password
protect your admin pages so cool everybody have a great rest of
your week and uh we'll see you on the community hopefully we'll see you on Friday at the hangout take care everyone bye