Rules to Better Networks
Networks are the lifeblood of any business. This is why we have developed these standards for better Networks.
If you still need help, visit our Network Architecture consulting page and book in a consultant.
For unplanned outages, see Do you have an unplanned outage process?
If your servers are down or have to go down during business hours you should notify the users at least 15 minutes beforehand so you will not get 101 people all asking you if the computer is down.
For short outages (under 15 minutes) that only affect only a few people (under 5 people), or are outside of business hours, then IM is the best method. If you use Microsoft Teams or Skype, a quick message will do.
Note: If they are not online on Teams or Skype, then they can't complain that they were not warned.
For extended or planned outages, or if you have a larger number of users (50+), email is the suggested method.
If you send an email it is a good idea to tell the user a way to monitor the network themselves. Eg. Software solutions like SCOM or WhatsUp Gold.
Include a "To myself". It gives visibility to others who are interested in what needs to be done to fix the problem and makes it easier to remember to send the 'done' email. E.g. "done - CRM is alive again".
To: SSWAll Cc: YYY Bcc: ZZZ Subject: This is the subject
Here is the summary of the outage plan:
Planned/Unplanned: Planned Change Description: Install Windows Updates and Restart Server Risk (see table below): LOW RISK (LOW Probability and MEDIUM Impact) Reason For Change: Windows 2016 Windows Updates Uptime over last month: 91.361% Planned Outage (mins): 150 Planned Start Time: 26 October 9:00 PM Planned Finish Time: 26 October 11:30 PM Affected Services: \Windows Server 2016 , sharepoint.ssw.com.au intranet.ssw.com.au , projects.ssw.com.au
Risk Lookup Table by Probability and Impact:
Note: The following servers will be affected:
To show others who are interested in what needs to be done to fix the problem:
Detailed Change Plan:
- Lockout users via IIS
- Backup server
- Install Windows Updates
- Reboot server
- Follow test plan
- Based on result of test plan, follow backout plan if procedure failed
- Procedure completed
- Check Event log for errors
- Check each affected service is running
- Call test users to start “Test Please” on the affect services
- Get result of user “Test Please” by email by 11:15 PM
- Restore server from backup
Note: <This is as per rule What is your server reboot/restart policy? >
Immediately before the scheduled downtime, check for logged in users, file access, and database connections.
Open 'Windows Task Manager' (Run > taskmgr) and select the 'Users' tab. Check with users if they have active connections, then have them log off.
Open 'Computer Management' (Run > compmgmt.msc), then 'System Tools > Shared Folders'. Check 'Session' and 'Open Files' for user connections.
Open SQL Server Management Studio on the server. Connect to the local SQL Server. Expand 'Management' and double-click 'Activity Manager'.
Once these have been checked for active users, and users have logged off, maintenance can be carried out.
Restarts should only be performed during the following time periods
- Between 7am and 7:05am
- Between 1pm and 1:05pm
- Between 7pm and 7:05pm
If a scheduled shutdown is required, use the PsShutdown utility from Microsoft's Sys Internals page.
Always reply 'Done' when you finish the task.
We recommend enforcing strict password policies.
Below is a capture of the settings we use:
When passwords have to be changed they must meet the following minimum requirements:
- Not contain all or part of the user's account name
- Be at least six characters in length
Contain characters from three of the following four categories:
- English uppercase characters (A through Z)
- English lowercase characters (a through z)
- Base 10 digits (0 through 9)
- Non-alphanumeric characters (e.g., !, $, #, %)
Rember it is always good to use an even number for password length ;) https://www.troyhunt.com/how-long-is-long-enough-minimum-password-lengths-by-the-worlds-top-sites/
Complexity requirements are enforced when passwords are changed or created.
Every 180 days clients will be required to change their password, they can change it when:
- Login to their computer
- Terminal server to another computer
This allows users to change their password by making a VPN connection to the office.
We also enforce a lockout policy so if a user gets their password wrong 5 times, their account will be locked out for 15 minutes.
If you want to change your password sooner, press [ctrl] [alt] [delete] then click "Change Password" button.
For planned outages, see Do you have a server reboot/restart policy?
During your course of being a SysAdmin, you will come across many outages. Some of them will impact BAU (Business as usual) and others will just be minor service outages. Do you know what to do you have a plan in the event of these outages?
Below is a process for these types of outages. Some amount of common sense is required here, an outage would be if services that would affect BAU work are disrupted for SSW and/or some hardware has failed.
- Blade Servers
- SAN Storage
- Active Directory Domain Services
- O365 Services; Teams, SharePoint, Exchange, OneDrive
- File Servers
- SQL Servers
- IIS Servers
Determining What Services are Disrupted
At SSW we use WhatsUp Gold for our device monitoring, however, there are many tools for this, Solarwinds, SCOM, etc
- Login to monitoring service
- Check to see what services are down
After you have determined what services have been disrupted it is time to call your SysAdmin team and organise a quick conference call. This will allow you to have a discussion prior to making any changes/fixes that could cause the outage to become worse.
Key Discussion Points:
- What services have been disrupted?
- What is the impact of these services?
- Is an email to everyone in your company required?
- What are your next steps?
What if you cannot reach anyone?
If you cannot reach anyone move onto the Email section.
If from the previous discussion you have determined that an email needs to be sent to your entire company, or you have decided this is necessary if you cannot contact anyone above, send an email in the following format:
To: SSWAll Subject: SysAdmins – Outage Notice
We are experiencing an outage and the following services have been affected:
We are working on restoring these services and will keep you updated.
Thank you, SysAdmins
A separate email needs to be sent to SysAdmins outlining what was discussed on the call. If no one was contactable, please proceed with what you have determined on your own.
To: SysAdmins Subject: SysAdmins – Outage Notice
As per our conversation,
The following services are disrupted:
The impact of these services disrupted are:
ZZZ We have decided that an email to ALL is/is not required.
The next steps to resolving this are:
Thank you, SysAdmin
Next steps did NOT resolve the issue
If you have completed your tasks but the issue has not resolved, please try to make contact with the SysAdmin team again and send an updated To Myself email.
Next steps resolved the issue
If your actions have resolved the issue, please notify ALL of the services being restored and update your To Myself email.
Do you protect your users and administrator accounts with more than one authentication method?
What is Multi-Factor Authentication (MFA)?
MFA is another layer of security for your users and administrators, it adds another 'password' or code that you can receive in a device that you possess - a phone, for example - to make it more difficult for attackers to steal your account.If they guess or brute-force your password, they still need the second code to make it to your account.
Generally, every time you log in on a service, it will ask for your normal password and an additional code. This code can be retrieved through an authenticator app, through an SMS, or even through a phone call to your mobile.
It is best practice to apply MFA to your Administrators first, as their accounts are the most important on the company and have access to all resources, and your users second, which still benefits from added security.
Do you have Password Writeback enabled in your Azure AD Connect?
If you want to let your users reset their own, on-premises passwords directly from the cloud, you need to have Password Writeback enabled in Azure AD Connect!
You can read more about Password Writeback from the Microsoft Documentation: https://docs.microsoft.com/en-us/azure/active-directory/authentication/howto-sspr-writeback
When setting up Azure AD Connect, you need to set the "Password Writeback" option:
Good Example: Setting up Password Writeback in Azure AD Connect
Of course, all your servers are on UPS. (If not they should be!) How do you know that all the money you paid for a UPS was worth it thought? How many times has it saved our servers? How long do the battery's last for before they go flat? Why was a server off when you came in in the morning?
If you get your UPS to email you when an event occurs then you will have answers to these questions.The problem is that there is no uniform software that will work with all UPS's as they all have their own format.All UPS's come with management software that can perform these actions. You just need to install it.
We use an MGE UPS so we use Personal Solution Pac which allows you to run script files on events. We just call a script file which will send us an email.
How often do you find files on your network file server that clearly shouldn't be there? Developers are notorious for creating temporary files and littering your file system with them. So how can you identify exactly who created or modified the file, and when?
The easiest way is to configure Windows file auditing .
Thankfully, Windows Server come with built-in file auditing. Any changes create and delete can be logged to your system event log. Here's how to set it up.
How to implement auditing on your file server
- Terminal Server into the file server
- In Windows Explorer, locate the directory you want to configure logging for (e.g. C:\Inetpub\wwwroot for logging changes to your website files)
- Select Security tab | Advanced
- Click the Auditing tab
- Select the users whose usage you want to monitor (usually all users, so select Everyone )
- Select what you want to monitor. For best performance, we only tick the options in shown in the figure below - there's no need to log when someone opens a file.
- Click OK and OK again to apply the changes. The process may take some time depending on the number of subfolders and files selected. Now you need to configure the system event log.
- Open Control Panel->Administrative Tools->Event Viewer
- Right-click the Security node and Control Panel | Administrative Tools | Event Viewer
- Right-click the sure Overwrite events as needed is checked
Checking who created the file
Now test to see if auditing is working.
- On the server, create a file called "test.aspx" somewhere in the path that is being audited
- Open Control Panel->Administrative Tools->Event Viewer
- Select the Security node, and notice the entries that have been created. They will have a similar format to the figure below.
That's all! It is also great for finding out who accidentally deleted files from the file system.
Furthermore, we can dump the event log to an Access or SQL Server database to make it easier to handle. Here is how to do it:
- Download the scripts: one for Access database and the other for SQL Server.
- Find and change the strEventDBConn variable to your connection string, also, modify strEventDB and tblEvents variable to your database name and table name.
- Write down the names of the servers to monitor in EventHosts.txt.
Done, now you need only double-click to start it.
This script is originally from http://pubs.logicalexpressions.com/pub0009/LPMArticle.asp?ID=340.
Previously the way we managed our certificates was using a SharePoint list as well as calendar reminders to inform us when they were going to expire. The issue with using this system is the SharePoint list as well as ensuring the certificates remained up to date was a manual process. This left a lot of room for human error especially when managing hundreds of certificates. There are of course commercial solutions to manage certificates but these haven't been econmical for our environment.
With Certify the Web and Let's Encrypt, we remove this human error and manual handling, ensuring that our certificates never expire.
You should use Certify the Web.
Do you know if your computer should be joined to the domain or not?
Joining your company's domain is a trade-off. If you join the domain, the company is the one responsible for managing your device, so all company rules and policies will be applied to it (Windows Update frequency, users, password resets, etc) and you will need to go through your SysAdmins if you have troubles with it.
If you choose to not join the domain, the machine management is all yours, giving you more freedom on the machine, but any automatic scripts would need to be done manually.
Below some pros and cons of joining the domain:
| | Pros (+) | Cons (-) | | --- | --- | --- | | Machine Management | Client management through GPOs (Group Policy Objects) | Lack of autonomy | | Resource Access | Direct access to resources (e.g. fileserver) | Needs to sign in first, or be attached to a VPN or the network to access resources | | Automatic Scripts | GPOs apply automatic scripts like the Login Script and Backup Scripts | Need to run Login and Backup scripts manually | | Support Level | More support from your SysAdmins, you have someone to rely on for any troubleshooting on all computer applications | Less support from SysAdmins, you can run any obscure application on your computer but that may not be supported by your company | | | | |
You need to log a record on success so you can check for backups that have failed.
Now you are able to be aware of missing backups. You can make automatically notification based on above table e.g. by SQL Reporting Services data-driven subscription
It is important that the network administrator can easily find out how reliable his servers are. This can be achieved using tools like [What's Up Gold] (https://www.whatsupgold.com) to monitor the uptime and SQL Reporting Services to create a report showing server uptime.
Here is a report that we use to monitor our servers on a daily basis:
Wireless networks are everywhere now. You can't drive down the street without finding a network which is insecure. However, in an office environment, there is a lot more to lose than a bit of bandwidth. It is vital that wireless is kept secure.
WEP, No SSID broadcast, allowed MAC addresses are all OK but these are more home security.
Figure: Bad example - the above settings are not suitable for a company's wireless access point
For the office, you need something a bit more robust and not requiring much management overhead.
It is recommended to use Radius authentication to integrate with your Active Directory.
Figure: Good example - configure your wireless access point to authenticate against AD
This article explains how to setup your wireless AP to use WPA2-enterprise. WPA2-Enterprise verifies network users (AD a/c's) through a server (Domain Controller).
The recommended method of authentication is PEAP (Protected Extensible Authentication Protocol), which authenticates wireless LAN clients using only server-side digital certificates (In our case we used an AD CA) by creating an encrypted SSL/TLS tunnel between the client and the authentication server. The tunnel then protects the subsequent user authentication exchange.
- 802.1X-capable 802.11 wireless access points (APs)
- Active Directory with group policy
- Network Policy Server (NPS) servers
- Active Directory Certificate Services based PKI for Server certificates for NPS computer/s and your wireless PC's
This document assumes you have some knowledge of how to configure your wireless access points and install server roles. It also assumes that you have already configured an Enterprise Certificate Authority on your Active Directory Domain.
- Figure: Unifi AP1. Broadcast the SSID 2. Use AES-CCM Cipher for encryption 3. Use WPA2 (mandatory) 4. Use the native VLAN 5. Configure a shared secret that you will use with the NPS Radius server Recommend using WPA2-EAP with AES as your encryption cipher.
Install NPS on your server On Windows 2008 or 2008 R2 open up the server manager and:
- Add the "Network Policy and Access Services" Role Under role services add:
- Network Policy Server
- Routing and Remote Access Services
- Configure Radius Clients on NPS Open up the NPS Console. Right click on "Radius Clients", and then click on "New". Fill out the fields for Friendly Name (enter the name of the wireless access point), Address (IP address) and then add the shared secret (Keep this safe for example we use Keepass as a password repository) you configure on your access point.
Configure 802.1x on the NPS server In the NAP servers Server Manager, open "Roles", then "Network Policy and Access Services" then click on NPS (Local). In the right-hand pane under standard configuration choose "Radius Server for 802.1x Wireless or Wired Connections", and then click on "Configure 802.1X" to start a wizard-based configuration.
- Select the top radio button “Secure Wireless Connections" click next
- On the Specify 802.1X Switches Page check the AP's you have configured under Radius Clients are in that list then click next
- Now the authentication method. From the Drop Down lists select Protected EAP (PEAP) NOTE: This method requires a Computer Certificate and the Radius Server and either a computer or user certificate on the client machine
- Select the groups (eg. Domain\WirelessAccess) you would like to give wireless access to. You can do this by user or computer or both
- If you need to configure VLan's in the next step, wasn't required in my case I just used the defaults
- You then need to register the server with Active Directory. So right click on NPS (local) and select Register Server in Active Directory
You should now have a Connection Request Policy and a Network Policy. Remove the MS-CHAP v1 authentication method from the network policy (under the constraint's tab).
Configure Certificate Auto enrolment First open Group Policy Management.
- Create a new GPO policy and name it "Cert_Enrollment_Wireless" or whatever name you deem suitable and link it to the root of the domain or a specific OU depending on your needs and OU structure
- Under the security filtering scope for what the policy gets applied to remove "Authenticated Users" and add your AD created. This ensures that the policy, once configured, is applied only to members of those groups.
Edit the settings of the group policy and go to:
- Computer Configuration\Policies\Windows Settings\Security Settings\Public Key Policies In the details pane, you need to right-click the Certificate Services Client – Auto-enrolment and then select properties. In the Properties, dialog box select enabled from the drop down box and then place a tick in all the remaining tick boxes. This makes sure that the computer auto-enrolls for a certificate from AD CA.
- Now navigate to Computer Configuration\Policies\Windows Settings\Security Settings\Public Key Policies\Automatic Certificate Request Settings Right-click in the details pane and select New | Automatic Certificate Request. This will open up a wizard and you can select a Computer Certificate.
Creating a Windows Wireless 802.1x GPO Policy
- Now go to Computer Configuration\Policies\Windows Settings\Security Settings\Wireless Network (IEEE 802.11) Policies Right click and Create a new policy for Windows Vista and later (if you only have XP machines, do only an XP one). If you have Vista or later you must do a Vista policy or else Vista will try to use the XP policy (not recommended).
- Enter a Policy Name (e.g. Beijing_Wifi_Settings) and description and link to the root of the domain.
3. Click "Add" and then enter a Profile Name and then Add the SSID name from the Wireless Access Point/s. Make sure the tick box "Connect Automatically when this network is in range" is ticked... 4. Click on the Security Tab Make sure Authentication is "WPA2-Enterprise" and Encryption is "AES). Under "Select a network authentication method, choose "Microsoft: Protected EAP (PEAP). Under Authentication Mode, you need to choose whether you want to authenticate computers and/or users with your digital certs. Then select "Computer Authentication". 5. Click on the "Properties" button Tick "Validate server certificate" and then tick "Connect to these servers". Enter the FQDN of the NPS. Then under Trusted Root Certification Authority, tick your Root CA certificate. Then click OK.
6. Click OK twice. Optional: Under Network Permission tab you can use the tick boxes to restrict clients to infrastructure networks or only GPO profiled allowed networks if you desire. 7. Click OK and you have completed your Windows Wireless Policy
When guests come to an SSW Office, we provide them with easy Wifi access using a QR code. This saves people manually typing in a password and can have them up and running in a matter of moments.QR codes can easily be created with services like QR Code Monkey.
What is the best option for your business when it comes to securing your website with HTTPS?
When you create a website, you can only access it through HTTP (http://), and not securely through HTTPS (https://) if you do not own an SSL Certificate.
When it comes to website certificates, you can choose from free or paid SSL certificates!
Free certificates can be obtained from Certificate Authorities like Let's Encrypt, which is helping provide free and automated certificates for the web.
- provide the same level of SSL encryption as paid certificates;
- provide HTTPS with a green padlock on the address bar of your browser, just like paid certificates;
- can be automatically renewed, easily.
Why would anyone use paid certificates, then?
If you are operating a big business, paid certificates give you some more assurances over free ones, and you can obtain them through reputable Certificate Authorities like Comodo, GeoTrust, Symantec, etc:
- gives you warranty against misuse or wrongly issued certificates;
- are normally valid for at least 1 year or more, while free certificates are only valid for 3 months;
- offer support for any errors or problems you have with your certificates.
SSL Certificates are an important part of any reputable website, so if you are operating a small website, blog, testing environment, personal site, anything that doesn't need too much support, getting a free certificate is the way to go.
If your business or site does not fit on the above affirmation, getting a paid certificate is the best option!
Do you use service accounts for recurring tasks and systems, or do you use user and personal accounts?
As a rule, you should never use a user account for accessing systems, reports, tasks and other long-running applications that do not need human or user interaction to run.
Service accounts provide a security context where the applications run, without the need to worry about their passwords or privileges. If a user changes their password, you will not break anything, because service account password normally does not expire and changing them is never needed.
Also, if the security of a user account is breached, you do not have to worry about any other systems being compromised - that account was not being used to run any applications. Always keep your service accounts passwords safe and complex, and you will never need to worry about it.
Occasionally, one server and its drives will not have sufficient space to store all related files in a network share. For example, you may have a "SetupFiles" directory that stores all Setup executables on your network e.g. \bee\SetupFiles. There are problems with this approach.
- You will run out of space - which means you will have to copy or move old (but still used) setup files around to other drives (\bee\d$\SetupOld\ ) or other machines e.g. \tuna\SetupFiles. This fragmentation of your setup files can cause confusion for your users.
- When you retire or rename the old server, links to the old server location will not work
So how do you get around this problem? The answer is in the Distributed File System (DFS) . Instead of having several server-specific file share locations, you can have a domain-wide setup location that offers a seamless experience to your users. DFS will even track a history of when and where file locations were moved.
A company-wide template will be implemented, so users have automatic footers to save time and give better branding.
How to have a company-wide Word template:
- Modify your Normal.dotm file to have the headings and format that you want for Word document
- Create standard employee email footer files e.g. JamesZhou.htm or JamesZhou.txt
- Put the files on a network location - this is the place that will have the master copies
- Have a logon script which is set up through Group policy that will copy the file to the users' computer when they logon. e.g. a PowerShell login script like https://github.com/SSWConsulting/SSWSysAdmins.LoginScript
ECHO Copy Office Templates To Workstation >> %LogonLogFile% call %ScriptFolder%\SSWLogonScript\BatchScript\SafeCopyNewerFile.bat "\\fileserver\DataSSW\DataSSWEmployees\Templates\Normal.dot" "%APPDATA%\Microsoft\Templates\Normal.dot" %LogonLogFile% call %ScriptFolder%\SSWLogonScript\BatchScript\SafeCopyNewerFile.bat "\\fileserver\DataSSW\DataSSWEmployees\Templates\Normal.dotm" "%APPDATA%\Microsoft\Templates\Normal.dotm" %LogonLogFile% call %ScriptFolder%\SSWLogonScript\BatchScript\SafeCopyNewerFile.bat "\\fileserver\DataSSW\DataSSWEmployees\Templates\ProposalNormalTemplate.dotx" "%APPDATA%\Microsoft\Templates\ProposalNormalTemplate.dotx" %LogonLogFile% call %ScriptFolder%\SSWLogonScript\BatchScript\SafeCopyNewerFile.bat "\\fileserver\DataSSW\DataSSWEmployees\Templates\NormalEmail.dot" "%APPDATA%\Microsoft\Templates\NormalEmail.dot" %LogonLogFile% call %ScriptFolder%\SSWLogonScript\BatchScript\SafeCopyNewerFile.bat "\\fileserver\DataSSW\DataSSWEmployees\Templates\Microsoft_Normal.dotx" "%APPDATA%\Microsoft\Templates\Microsoft_Normal.dotx" %LogonLogFile% call %ScriptFolder%\SSWLogonScript\BatchScript\SafeCopyNewerFile.bat "\\fileserver\DataSSW\DataSSWEmployees\Templates\Blank.potx" "%APPDATA%\Microsoft\Templates\Blank.potx" %LogonLogFile% xcopy /Y "\\fileserver\DataSSW\DataSSWEmployees\Templates\NormalEmail.dotm" "%APPDATA%\Microsoft\Templates\" >> %LogonLogFile% xcopy /Y "\\fileserver\DataSSW\DataSSWEmployees\Templates\NormalEmail.dotx" "%APPDATA%\Microsoft\QuickStyles\" >> %LogonLogFile% ECHO Templates Copied
Figure: Bad Example - This is a snippet of an old login script
You can automatically have your SSW Word doc template on sign-in via a script. See https://github.com/SSWConsulting/SSWSysAdmins.LoginScript
Good Example - New Login script on Github
Note #1: We don't want people using .RTF emails so we include this message in SSW.rtf. Be aware that we don't want to use RTF because of Remove RTF as an option or explain when it is a good choice.
Note #2: If you use a Mac computer, a login script will not work. In order to use a Word template, you must open the template on Word locally, hit "Save as Template", and then upload that document to Teams.