A new customer called me and mentioned he was being billed for calls that he wasn’t making on his Asterisk based PBX system. I knew right away that his system had likely been compromised, and this wasn’t anything out of the ordinary for us to tackle. As I dug in, it looked to be an unpatched version of Elastix that had a simple compromise, and someone was using it to make phone calls. The system was reconstructed, and is now back in working order. However, what I also found during the investigation is the asterisk configuration was set to record inbound …
Here at MNX Solutions, we're busy setting up a brand new data center for our hosted services - MNX.io. A large part of our business involves consulting, which means we're exposed to a ton of different customer environments and an equal number of schemes for naming equipment...not all of them good. It's a problem that goes back as far as computers have existed, and everyone has their own opinion on the "best" way to name hosts. Most methods start out fine at the beginning, but quickly become unwieldy as infrastructure expands and adapts over time. Since we're starting fresh with this …
This is a guest post from Mark Stanislav of Duo Security -- At MNX we use Duo everywhere we can, and we think you should too. If your not using 2FA, talk to Duo and start securing all of your password logins! The conversation always starts the same way, "I think my web site was hacked." Recently, a friend of mine brought this topic up and I immediately went into incident response mode. After checking out his evidence of the alleged breach, I quickly noted multiple known vulnerabilities afflicting his various WordPress deployments hosted on his server. In his case, he …
While working on a monitoring agent for a service we will be introducing soon. I needed a way to determine the Major/Minor numbers for a device (e.g. /dev/mapper/datavg-srvlv). It is actually much easier than I though. My first version as a gist. But a much simpler solution exists, without using cgo. stat := syscall.Stat_t{} _ := syscall.Stat("/dev/sda", &stat) fmt.Println("Major:",uint64(stat.Rdev/256), "Minor:",uint64(stat.Rdev%256))
As most geek discussions go.. we like to push the envelope -- to see what is possible! As we discussed using chattr to protect a file from being overwritten, it occurred to me that we might perform a weak form of "Inception" on chattr! So I ran: # chmod 000 chattr & chattr +i chattr # chmod +x chattr chmod: changing permissions of `chattr': Operation not permitted # chattr -l -bash: /usr/bin/chattr: Permission denied # chattr -i chattr -bash: /usr/bin/chattr: Permission denied # chmod +x chattr chmod: changing permissions of `chattr': Operation not permitted And one solution To restore functionality to chattr ... # cp /usr/bin/chattr /usr/bin/chattr2 # chmod 755 /usr/bin/chattr2 # chattr2 -i /usr/bin/chattr # chmod …
We had a support request recently that indicated a server was under heavy load due to a wordpress wp-login.php brute force login attack. This attack was impacting 3 customer servers from more than 500 different IP addresses. We needed a solution fast. Others on the internet have reported tens of thousands of unique IPs involved in this attack. There is currently a significant attack being launched at a large number of WordPress blogs across the Internet. The attacker is brute force attacking the WordPress administrative portals, using the username "admin" and trying thousands of passwords. It appears a botnet is …
When you want to develop a new Puppet module, it's best to work outside of your production environment. You could use Puppet's built-in environments feature to slice up your site, but there are a handful of caveats and it's not always the most convenient. Enter Puppet Sandbox... Puppet Sandbox is a multi-VM Vagrant-based Puppet development environment that can be used for creating and testing new modules independent from your production servers. It's also a great way to get your feet wet if you're new to the whole world of Puppet and automated configuration management software. While you could manually configure this type …
While working on a postfix/dovecot non-mysql virtual domain configuration, I needed a simple admin script to add new users. The script below is what I came up with after about 30 minutes. I was unable to find something that met my needs on the interweb, so I hope this post finds someone else in need! Please submit bugs/suggestions to: nick.wilkens@mnxsolutions.com [code] #!/bin/bash # bugs/suggestions to: nick.wilkens@mnxsolutions.com # 1/29/2012 USERSFILE=/etc/dovecot/users POSTFIXVIRTUAL_MAILBOX=/etc/postfix/virtual_mailbox_maps POSTFIXVIRTUAL_DOMAINS=/etc/postfix/virtual_domains function validate_username() { username=$1 echo $username| egrep -iq '([[:alnum:]_.]+@[[:alnum:]_]+?.[[:alpha:].]{2,6})'; RC=$? if [ ${RC} -ne 0 ] then echo "Invalid username, please ensure user@domain.tld format ($RC)" exit 1 fi } function get_username() …
When working with a client, we were discussing storage for an application they have developed. Storage price was a key component for success of this product. I read the BackBlaze posts here, and here -- but I wanted a mix of cost savings, off the shelf parts, and hot-swappable drives. These systems were built in October when we were able to purchase 3TB disks for $129. With the current disk prices, the total cost of the storage node is now about $11500, based on $219 3TB disks. The client needed to store data for archive purposes. They planned on deploying multiple nodes, …
As is probably obvious from our blog posts on Yubikey SSH/WordPress two factor authentication and Google Authenticator SSH two factor authentication, we at MNX Solutions are big proponents of multi-factor authentication schemes to add security to infrastructure. On the other side of things, though, we have to be agile system administrators in order to offer customers efficient and timely service when a problem occurs. While we still really enjoy our Yubikeys (especially for WordPress), we've found that the offering from Ann Arbor, MI based Duo Security is the one for us. Numerous Points of Integration Directly related to us, Duo easily integrates …