It may be a change in management or even an emergency, like a worldwide pandemic. It could be a couple of employees taking their PCs home, or hundreds of them. The fact that remote work is becoming mandatory for a large percentage of people can be dangerous for any organization, big or small, prepared or not prepared. 

And not really dangerous because of its nature –after all, here at Prey we live and breathe remote work– but because operations, regardless of their scale, have to keep running with no failures. That’s where our IT managers and Sysadmins excel, specially in this COVID-19 crisis. A world where it’s becoming difficult to work together can be a nightmare for the ones who secure and manage the devices we use to work.

In the corporate world, there are a couple of concepts that usually go together: Business Continuity and Disaster Recovery. On one hand, Business Continuity is all about keeping the show running, regarding any factor that can disrupt our day to day work; on the other hand, Disaster Recovery cares about the vital infrastructure of your business’s operations. Both go side-by-side when disaster strikes. And while DR can be somewhat of a hassle on a viral crisis like the one we’re having today, the plan every CIO or IT management office is focused on right now is Business Continuity. It may be even worse: there could be no plan.

In that scenario, where a large number of people have to work from home unexpectedly, is where questions arise. How are we managing our remote teams and equipment? How about our servers? Our security? Our access to a private network? What about Jim from Accounting, who has a huge desktop PC and can’t take it home? Even one of those questions can be a nightmare for any admin out there and has several ramifications over security and management. We know. We’ve been there.

That’s why we’ve compiled a very general guide on those topics we -and some other angry, stressed Sysadmins on Reddit and other platforms- think are important to keep around: because things are going to get bad very, very soon.

Best case scenario, you become the hero of your office; worst-case scenario, everything crashes inevitably, but at least you thought about it beforehand. Not sure you’re getting your photo framed in that case, though.

Can Staff Work Remotely?

This should be on your checklist first and foremost. Modern offices have a myriad of professionals: management, development, finance & accounting, design, marketing and so on, so their needs regarding devices are very broad. Although, if you leave out of the equation the ones with specific technological needs (no Brad, we can’t relocate the office’s plotter printer to your apartment) you may realize your remote workforce is very homogenous in needs. We’ll handle equipment later; first, you have to think about remote access, securing devices, and physical barriers to critical equipment.

Is your system built to support remote access? Most IT managers are used to physical configurations: a secure network, policies, RDP/VNC (to access and handle a device remotely) and communication, all between computers usually connected to the same network. ¿What happens when everyone takes their device home? You lose the privileges of controlling the network from the inside, so everyone –especially you– grabs their hair when even a minor change needs to be done. 

The best advice for this, given the time constraint, is to limit as much as possible who can connect to your secure network, and always using encryption protocols -such as SSH- if you’re dealing with sensible information on remote work devices. 

The same principle can be applied to third party solutions to remote access. For example, if you want to set up a VNC alternative –like the dreaded Teamviewer– make sure it’s at least on a secure connection on both ends, and totally encrypted.

Is there enough capacity for the remote access needs of your employees? Chances are if you’re dealing with devices on a medium or large scale organization, you probably have a secure VPN for any remote users to connect to. Likewise, that VPN may come from known providers, like Cisco or Citrix, who sometimes vary their fees based on the number of devices connected or the total bandwidth. Keep that in mind.

Speaking of bandwidth, and especially if you have servers or critical devices there, is the office’s internet connection capable of sustaining multiple connections from the outside? A short call with your ISP can solve a lot of issues moving forward.

Are your employees connected to the internet at home? We know: internet access is kind of ubiquitous at this point. Nevertheless, certain ISPs around the world deliver substandard connections, which can put a grapple on any demanding service needed by any remote work environment, like videoconference, file transfer or even VPN connections. Make sure your end-users have a stable wired connection, or make space in the budget for untethered internet, like 4G dongles, for key users.

Do you need physical access to your servers? Most IT managers have workarounds about this, especially in terms of virtualization: most racks can be managed remotely and can be scaled accordingly. Although, others may have a different problem: backups. Some data centers and law-compliant enterprises have LTOs: machines that backup critical information on high capacity tapes (yes, tapes!).

Most tape-reliant backups are manual, so you have to remove one or several tapes from the rack and store them on a secure location, usually a safe. If your organization relies on this backup method -and if you’re the one in charge- probably you have a protocol for this. Due to our recent viral circumstances, you could still be the chosen one to perform that task.

In any other case, and if you’ve been remotely managing servers for years, go home. And if your boss says otherwise, you can always ask him for a little help like this sysadmin on Reddit. 

Does your staff have everything it needs to work remotely?

This is a cornerstone of remote work: how can we work from home if we don’t have the equipment to do it? In most cases, this equipment comes from the office, so several –or all, for that matter– employees got to work on Monday, took laptops, monitors and accessories, and drove home. In other cases, especially in small businesses, there may be a ‘Bring Your Own Device’ approach: work with what you have at home. We’ll tackle the first one, since there is no sure policy on BYOD, particularly on zero trust models.

In the office equipment scenario, the two most concerning questions are clear: Do we have enough devices to satisfy demand? And second, are we securing the ones we have?

Concerning device demand, it’s important to be as transparent as possible with device and software acquisition, spare parts and repairs. Does everyone have a device to work with? If you don’t, someone should buy the needed equipment. Do you handle that budget, or someone else? You need to work very closely with that person and/or team, to ensure everyone has the tools they need. Of course, there are a couple of loose cases where management will acquire devices without consulting the IT manager. Nevertheless, if there’s any kind of issue with those devices they will seek your help anyway, so beware.

Speaking of help, there is a great chance that a lot of devices will be damaged over time. Wear and tear are pretty normal for any device, but a remote work environment can pose other threats. Drinks, pets, unorthodox usage –a badly ventilated laptop being used in bed comes to mind as an example– and household accidents are much more common, especially if the end-user isn’t very careful, or not accustomed to the remote work culture.

A rule of thumb for a medium or large organization: have one spare device for every ten workers. Make sure these replacements are available to your employees: it’s no use if our IT guy has all of them on the other side of town. And if you’re that IT guy, consider leaving at least a couple backup PCs in the office and getting a local permit to move around and help people. In most countries, total quarantines are being enforced by law, but probably -and depending on your location- you may ask for one based on the Business Continuity Plan of your organization. 

The same principle can be applied to spare parts. You still have time to hoard hard drives, RAM memory sticks, screens, and cables, so you can distribute them if trouble arises. 

As with devices, apps and data are also an issue. A lot of the present headaches in the sysadmin community are the lack of computer knowledge of most users, to the point that ITs are banging their heads against the wall trying to get an employee to update their firewall and permissions, mostly over the phone. We can’t stress this enough: send every PC absolutely ready. 

Consider it needs to face the challenges posed by a heavy –and sometimes brain-lacking– remote work user. That covers every nook and cranny: antivirus solution installed and updated, VPN configured, firewall set up, the apps needed by the employee installed and updated, and, if possible, a persistent rollback.

The last challenge of having devices for everyone comes with a price: as a systems administrator, IT Manager or even CIO, you have the responsibility of taking care of equipment on different levels. That’s where a more granular approach is needed, especially on teams that are still fearful of their employees working from home. For example, a small real estate firm wants to control where the devices are, at what times they’re being used, and how.

This issue can be solved with a Mobile Device Management solution, depending on the specific needs of your workforce. If the Administration tier only wants to check in on devices, networks they’re connected and usage, a lightweight solution is available. Otherwise, you may need pricier and robust options.

Can we maintain an acceptable level of cybersecurity?

Okay, we’re almost done! While the first two questions can be resolved by a vast majority of IT professionals, this is the one that requires the most patience and technical skill. Most of these issues are ubiquitous: for example, having an antivirus solution, as long as you’re not using Linux, should be required by law. However, there have been a bunch of questions and issues raising on forums and chats that can be a problem for you as well.

As such, we’ll list the most common issues we’ve seen, with their possible solutions.

Is my critical infrastructure secure from insecure remote workers? With a lot of devices going home, cracks begin to appear on the security dike. As such, everything must be checked beforehand: device encryption, identity and access management (IAM), file permissions and access policies. The use of a secure VPN is strongly suggested (see below).

From the end-user point of view, it’s critical for us to reduce any road a hacker can take to come inside our framework. Create a guide for your employees that includes storing passwords in a safe location, revisiting or improving their router security and an internet connection, and using Multi-Factor Authentication for most, if not all, remote work accesses. Establishing a Zero Trust Model on your organization beforehand is always a very safe step.

Is my employee’s internet connection susceptible to attacks? Extending the first case, you can take care of every imperfection on an endpoint, and it could still be insecure if the internet connection is insecure. The same concern can be applied to IoT –Internet of Things– devices. What if our organization gets hacked because of an intelligent light bulb?

The key to solving this is the router’s security. Most disasters can be averted with a strong WPA2 password and limited access to any IoT device in the employee’s home. If the employee handles critical information, you will need to go deeper: if you can, check the router for default logins, weird DMZ rules, and connected devices with strange MAC addresses.

Is it safe to control my employee’s computers remotely? As we said earlier, any RDP or VNC access through the internet –and not on a secure, internal network– is risky and considered a bad practice among sysadmins. Nevertheless, you will need a remote tool for servicing, patching, and even authorizing or performing new installations.

If you’re compelled to do it, be careful: always use encrypted peer-to-peer protocols, and limit your remote access to short sessions.

Are VPNs totally secure? Not by a long shot. Recent reports indicate that certain ports, associated with providers like OpenVPN or SSL VPN are at risk of being exploited: 1194 (OpenVPN), TCP/UDP 443 and IPsec/IKEv2 UDP 500/4500 (SSL VPN). Over the coming days, check the logs to see if those ports are not accessed or compromised by malicious individuals.

Is Multi-Factor Authentication worth the investment? If any kind of breach would risk your business, it’s imperative you use MFA for everything you can. Multi-Factor is the use of several ‘factors’ to prove your identity to a server, meaning every time an employee logins, specially when doing remote work, it’s him and not somebody else.

There are several auth providers that support MFA, using free tools like Google Authenticator, proprietary mobile services, and other factors like SMS, emails and verification codes. Just make sure your employees know how to use it, so they don’t fall to any social engineering scam.

Am I breaking compliance if we go remote? You would, but you shouldn’t. Depending on what certification we’re referring to, the requirements can be different. For example, SOC 2 needs strict web filtering measures that are easy to control on the office, but harder to block when everyone’s working remotely. Having said that, there are several Web Application Firewalls (WAF) that can be installed and configured on devices beforehand, so to ensure compliance. Chances are, you did most of the work already trying to get certified.

Takeaways

Most of what we’ve compiled on this guide may be general knowledge for most sysadmins. It’s possible that you’ve read this entire piece and said: “but I know this!”. And still, even the keenest people have to be reminded: securing devices is hard, lonely and sometimes ungrateful.

The world is amidst a huge pandemic right now, and it’s times like these where the heroes behind the curtain step to the remote work challenge. Being a systems admin is a thankless job, so we hope these general steps for maintaining peace and order in these times of crisis are helpful. And hey, maybe show this to your boss so he can finally get an idea of how hard it is what you do.

Godspeed.

If we are to take some lessons out of the COVID-19 crisis we are going through, we can say that most companies around the world were not ready to flip the switch and go for a remote work culture instantly.

It’s a complex situation, and not all positions fit the remote work mold. However, those that can be adjusted to it will find the challenge of dealing with it for the first time.

Aside from technical details, such as picking Slack over another communication tool, or defining proper remote access protocols, there are some day-to-day communication hurdles remote teams must overcome to smoothen out a remote work operation.

These small communication issues start appearing with time, and they can mark the difference between a team who is up to date and communicative, and a disconnected array of one-man-band operations that don’t feel like a team at all. These are the sort of things you start hearing when they happen:

• Didn’t you hear about those changes?
• Wait, what project is X talking about?
• … Were we responsible for that?
• Who is X, anyways!? Why is he only online at 5 am?
• But Y told us to do the other thing!

Jumping Remote Work Hurdles at Prey

At Prey, we’ve had a hybrid-remote culture since the company’s conception, back in 2009. We’ve faced many hurdles along the way and have managed to create a location-fluid work environment, where no matter if you’re going to the HQ today, or moving abroad to an entirely different time zone, we can work together.

In 10 years we’ve jumped many hurdles. We ditched several management tools; we ditched emails for Slack; workers moved out, and we moved offices; we had to create guidelines for communicating, and faced many humane challenges along the way.

But, more importantly, we’ve had plenty of time to identify and jump these hurdles, so that they don’t surprise you. Let’s take a look at some of these and we’ll share our advice on how to tackle them.

Setup Availability and Collaboration Expectations

When your coworkers stay at home, the first trouble you will face is that there are many things that can disrupt their schedule. Especially in a sanitary crisis scenario, where not only they have to stay at home, but basically everyone else. Their family, roommates, pet of choice, etcetera.

So, from time to time, something will come up. Someone’s at the door… and the phone’s ringing! Also, the dog made a mess, and to top it all a member of the household needs help with something that’s really, really important.

Asking for a complete shutdown of the worker during work-hours will put stress both on the worker and his environment. Be flexible, and adaptable, because it will happen. Measure progress with short-term milestones, and not in time spent.

The other problem you will face is that when your crew is working remotely, you can’t simply glance over your shoulder to see if they are available or working on something else. So, to ensure your team doesn’t miss each and create bottlenecks, you need to set up expectations:

• Will you need a coworker’s help on a task this week? Need someone to review your work? Don’t surprise them with it, let them know in advance so that they can make time for it.
• Is your dog on the roof and you need to go offline to rescue him? Be extra communicative about your availability. A quick update ensures you don’t leave people hanging.
• Feeling your team is disconnected from day-to-day progress? Have sync meetings that accommodate your work’s pace. Maybe a daily call works for an agency doing campaigns, but a sales team might review goals weekly.
• Are you collaborating live on a project but feel like email/chat is too slow? Do live collaboration calls (like Agile’s war-rooms) to streamline decisions and changes.

Scrum is a good methodology to adapt

You can kick off this by adapting one of Scrum’s ceremonies, the daily -or standup- meeting to your area. This is a short, ideally 15-minutes, meeting when the day begins. Each team (by area) has a quick-sync call to see what are the tasks for the day, coordinate efforts, and raise flags for obstacles or needs, like dependencies!

If the mold doesn’t apply well to your area (it is designed for software development after all) take the check-in concept and build one that makes more sense to the area’s pace and type of work.

The main concepts from Scrum that translate well to any area are: plan your work in short bursts (1-4 week sprints), break down projects in iterations, follow daily progress, review the work done, optimize, and plan ahead for the next sprint.

Keep Updates, Changes, and Decisions in the Open

Following one of journalism’s commandments: never assume knowledge. When working in a remote environment, important information gets lost easily.

Sometimes someone couldn’t make it to a meeting, and he missed some important decisions; other times a private chat results in changes that don’t reflect previously agreed-on goals; or worse, we simply forget what we agreed on, and that’s when assumptions generate a lot of problems.

There’s simply not a virtual watercooler where the day-to-day chit chat goes about. So, in its place, you have to make one. Well, that or have some communication ground-rules set, whichever works best!

The Remote Team Communications Commandments:

First, all agreements, changes, or decisions that come out of meetings should be documented and made public.

Secondly, if you’re working on a collaboration hub such as Slack, keep work-related discussions public to ensure anyone can hop into the conversation and give input.

Thirdly, assign a task’s responsibility to one person only to avoid communication issues, like two managers making decisions in parallel. 

Fourthly, record calls and presentations if someone is going to miss it.

Fifthly, granularize tasks on work management tools like Jira or Asana to keep an eye on day-to-day progress and avoid micromanagement.

Make Time for Your Team to Connect and Share

The internet community is stressing this one well enough. The first thing you’ll notice when going from on-site work to an entirely remote team is that you’ll hear and see a lot less from your coworkers.

Some might find it a relief! After all, positive or negative, a distraction is a distraction. For others, they might feel a bit disconnected and miss the jokes and daily little moments with their work-pals that fill in the gaps in between work.

Whether you’re one or the other, one thing is common grounds: you need to keep good chemistry with your team. It helps each other collaborate, it helps keep spirits up, and it will help everyone overcome the difficulties they find in remote work.

There’s a ton of choices in this department and there is room to get creative, but we’ll share our own activities to give an example!

TGIF Talks

This is our own version of TED Talks, which we hold internally biweekly on our end of sprint TGIF celebration! Anyone from the team can propose a topic, and prepare a presentation on it. Any topic that you’re passionate about, you can share! Here are some of the things we learned:

• How to grow and care for plants.
• The meaning of the Ubuntu concept.
• Day-to-day office cybersecurity

Playing Games

There is an endless catalog of games you can play online with your team. Whether you’re into charades, intricate board games, or video games, there’s a place for you online. The trick to making it fun for everyone? Keep it simple.

• Tabletopia is a great platform to play online board games.
• Charades can be played in video calls.
• Or create a weekly song-sharing tournament and have the teams vote!

Voice Chat Rooms

One day, we realized several of us had Discord installed, a free voice-over-IP communication app where voice ‘rooms’ are created and anyone can hop in and out anytime. Thus, our ‘virtual watercooler talk’ space was born.

Voice-chat rooms/channels like these are great to maintain that day-to-day chat and socializing that happens in the office.

It doesn’t matter if you’re muted for a while or need to hop off for a meeting. You’re just sharing a social space while you work from home, talking news, asking for feedback, and chatting a little about your week.

Groups will form naturally. Let it self regulate. After all, people share different interests and they most likely shared daily with those they were comfortable.

More importantly, you’ll need groups because this works great in cells of 2 to 8 persons. Above that, the conversation will likely be a bit chaotic.

---

Takeaways

There is plenty of resources out there on how to build a remote work infrastructure at your organization, but you should know it doesn’t end in the technical challenge.

Remote work takes getting used to, it will not be natural for everyone, and your communications will get mixed up if you do not take proper care.

However, from our own experience, we can reassure you it is not only worth the effort, but it’s also a necessary step forward to a healthier and more resourceful workplace. Take this chance to make it work!