Posted byu/SecureDigitsPlusLLC
22 minutes ago
Open Source Project | Freely Automated PXE Methodology | Research and Development | NOT MONETIZED

Secure Digits Plus LLC | Hybrid | Desired State ControllerMichael C. 'Boss Mode' Cook Date : Sunday June 6, 2019

My name is Michael Cook, and I have had the pleasure of 'repeating the same processes over and over again', and that would include, installing Microsoft Windows of all versions and flavors (as well as other operating systems but that's not the focus of my project yet), waiting for Windows to install, migrating Windows or performing an in place upgrade, transferring user profiles and their documents/settings/etc, downloading/installing applications/drivers, modifying the users profiles so that it works in the new environment and doesn't miss anything... but I can tell you, that even as meticulous as I am? I am still human and I make mistakes just like everyone else.

However, the mistakes I make are easy to fix when I service individual clients, it's just what you could call 'A pain in the neck' when something doesn't work, and I have to 'find out what I either “missed”, or, whatever program or application “didn't fit the protocol”'.Here's the dilemma. When you use something like the Media Creation Tool, you have to download it along with the updates, and the process involved there takes a lot of time because it downloads far more information and data than it actually needs to. ( It will report to the upgrading magistrate [you] what applications won't work, and will force you to confirm the removal of that application or software – but that's a side point )

The reason for this is because those updates aren't included in the image you download by default, it has to be an option so that people can test with it and otherwise make changes or alterations and validate that the new image works as planned. Doesn't always work as planned. Another problem is that when you download and then install updates, even on a machine that's already running, some of those updates require a reboot and a PXE update process in order to complete, for instance, a Security Rollup or what used to be called 'A Service Pack'.

When you utilize this method, you are essentially multiplying how long it would take to 1) download the image, 2) install the image, 3) download the updates, 4) install the updates, 5) depending on what types of updates you may need to repeat 3 and 4 a few times until you're up to date, and 6) the part that people most commonly forget to do? Disk cleanup.

Disk cleanup is ½ of the most important thing in order to retain security when updates are installed, and it is also a pretty slow process because it has to search through SIM in order to remove the files it doesn't need. That's what Disk Cleanup actually is... SIM - or at the very least, a division of what SIM is in entirety. That's what WinPE actually is, SIM. ( I would also argue that WinPE is a fancy version of 7zip... but that's a matter of speculation ) SIM is essentially their way of either 1) ensuring that legacy files that the operating system still relies on remain even after almost 15 years of not really needing them ? or 2) the only thing that is consistently bulletproof about their upgrade and/or imaging process. Some of those files that Windows doesn't need are under an umbrella called “WinSXS”, which is what CBS and DCOM are under the hood... and those files are what sysprep and unattend.xml require modifying in order to sustain integrity... but it's the long way and it's still not completely secure.

Michael Niehaus even explained this to me, hence the reason for my project.When you push an update that touches the Windows Kernel, the system has to reboot, AND, the files that were updated are left behind...? They're left behind on purpose. That's because Microsoft doesn't know whether or not their update process will work for YOUR machine in the long term... and that's because... a lot of applications rely on legacy modules and the update process “will break applications that rely on older frameworks that got replaced upon update.”Microsoft makes their very best attempts, but sometimes you get what happened with the 1809 update, where user's data gets wiped off of their machine completely, and then the customers or clients have to pay a company to completely restore their data.

If it happens, Microsoft won't help you recover that data... but if you try to sue them for their mistake, you won't be able to by yourself, and the maximum amount of money you'd be entitled to is a whopping '$5.00'. If you're like most people, it will cost you a lot more than '$5.00' to travel to King County, Washington to receive it, should you amazingly somehow win your case... I can tell you that most people will call that 'a huge waste of their time and money.' Not to mention, they make every effort to make sure this doesn't happen.

There are good engineers at Microsoft, and they do care about what they put out into the world, it's just that tending to billions of people is a massive undertaking and even as good as anyone can ever be? Mistakes will still slip through - so I chose to give them a break where they deserve one.

The process that I've put together, utilizes a method of entirely replacing the idea of using a WSUS server altogether, as WSUS will download updates and store them in a database for any given enterprise and then propogate those updates to each machine so that only the server touches the internet or receives the operating system updates, and does so securely - but slowly if using SCCM. At that point, a systems administrator or network engineer that does systems administration as well (like me), will have to sculpt out a group policy that ensures that the correct updates are targeting your machines, and you can use Active Directory to do all of this, or Security Templates, or GPEdit.MSC to do all of these things... but all of that requires a lot of 'time and money' (the hired expert who has to validate changes can be considered 'wages' and equates to the same thing here). It's not the creation of the template that takes the most time... That's the easy part. The hard part is iterating the process in as many scenarios as possible and making sure that errors are correctly handled. That's what validation means.

However, I can tell you, there's a better way to do things – through PowerShell and it's innate ability to access / change / update / manipulate registry values. But in addition to that, a better way to do things is, to inject the updates into the target image and then replace the WinSXS libraries within the distributed installation image, but ONLY after installing the updates from the Microsoft Update Catalog. DISM++ does an excellent job of grouping all of these things together... I'd go so far as to say, it's the most powerful tool in my arsenal when it comes to 'preparing all of my images with one tool'... it does what ICD, SIM, and WSUS all work together to do, and even a lot more than that.

The Microsoft Update Catalog is the link that every WSUS server uses, but instead of having to manually download the updates, the WSUS service has a digital signature or GUID that is able to enter the Microsoft Windows Update domain and pull the updates as they're needed. That's what every installation or instance of their operating system does when it automatically checks for and pulls needed updates.WSUS is useful to have, but it's a pain in the neck to maintain (ask any expert, they'll nod before you finish the sentence) and it's often the target of many hacks and exploits because those digital signatures are much weaker than the one Microsoft uses for their domain, which starts with b7... they're not meant to do anything more than just send instructions that change operating system files or replace them entirely.

Sort of like how regex is used to catalog changes you'd make on GitHub, or how you can 'undo' an action in Windows or whatever program and it will do that.Microsoft isn't inherently good, but it's not inherently bad either. There are some upsides to how they do things, as well as downsides, but every circumstance is different. There are perfectly legitimate reasons for wanting to design their products the way that they do, and even when they do make every effort to warn people about what they can or can not do with their products, there are still legal ways around their rules... as well as illegal ones that I don't care to perform. I understand many black hat approaches, it's just, a matter of my personal ethics that prohibit me from caring to participate or test those things, and, I'd rather help build a process which constructively solves the problems, not exploit them.

The part where black hats find entryways isn't really the employees, like programmers and engineers, faults... it's just part of the way their entire servicing stack is designed and it's a limitation they have to abide by all things considered... it probably makes their jobs a bigger pain in the neck than it needs to be, hence why there is a constant need to reassess their deployment and marketing strategies. However, when it comes to tracking every user that uses their software, with telemetry and DCOM, I can tell you that not all intentions there are pure... In fact, VSS and other aspects of DCOM are intentionally cryptic in nature because there are measures of control they want to sustain to achieve/sustain Quality Assurance, and 'to secure their intellectual property', that part is fine... but their choice of going about data collection leaves a gaping wide security hole in all of their products, and there are people that know how to manipulate those security holes even when they are 'patched out'.

Let me reel back in from the tangents I've gone on... The process I've put together is faster than any virtual machine based technology only because, when you're sending the images over a network, the operating system is already installed and expanded - if you're duplicating virtual machines on an existing machine or the network distance doesn't involve propogation through populated channels, then that method may be faster, the scenario matters, in the real world, you're not likely to have vSphere/vCenter or Hyper-V Server on every MDT or SCCM server, and in that case, any additional machines or systems you have to pass those images through will result in that process being slower. If you transmit the image itself..? You get what's called, 'The instructions to recreate that end result', and what this achieves is less transmitted data by offloading the work to the RAM, CPU, and HDD.

If you have a mechanical disk you're transmitting to, then no amount of optimization will really make the process any faster... I try to explain to people that differentials in a virtual machine environment still require changes and time to transmit, and alter the target machine state, but apparently, most people only care about how long it takes to go from 'not installed' to 'the desktop', and what they seldom take into consideration is 'making sure that all of the updates are installed, and the applied updates are cleaned up afterward, and that all of the drivers and applications are also installed and ready to go afterward'... which, people will easily dismiss the fact that using a VHD still means 'using sysprep / unattend.xml' which still takes time.

No matter what way you choose to do your updating, no matter what method you think is best, vectorizing the instructions and scripts takes up far less data than transmitting a completely installed image – which therefore requires less resources and infrastructure to transmit... Performing an upgrade is pointless if you can just do a clean installation and migrate the users existing data and installed applications structure, AKA application installation media - what happens is you bypass the need to go into a system upgrade state where you'll still have to perform disk cleanup afterward anyway.

Macrium uses a method that copies all of the data on a given hard drive, and, that method works for when you are changing a bad hard drive and want things to be 1:1, or maybe you want to expand or change the partition's size by expansion or contraction. PCMover uses a method that pulls user environment information from the registry, but whether you're using Macrium (or some alternative) or PCMover (or some alternative), there's no method out there that I know of that cleans the registry, removes malicious files or unnecessary files, transfers ONLY the user's profile and relevant data, makes an intelligent note of the applications most commonly used and their required dependencies, finds the application installation executable needed from the internet or a proxy, and then... parses out anything that's not within the ranks of the aforementioned subject matter - in order to restore a 2:1 end result for free.

Even if you use something like “Windows Repair”, or even Glary Utilities and various other tools that fall under the same umbrella... I can say that none of them do a 2:1 restoration of an existing user enviroment, and all of them cost $ and require licenses.I say 2:1, simply because, 2 is greater than 1, and you wouldn't be transferring data or profiles if there wasn't a problem or you weren't performing an update or upgrade... What you would get with a system I am attempting to build is this 2:1 end result... all of the aforementioned programs and tools will either 'cost money', 'clone the problems you were trying to remove', or 'not perform the end result you are looking to sustain'...

However, my process that I've explained seeks to only use clean installation images AND clean application images AND clean driver versions that get pulled from the internet, and all of these things are already ubiquitously deployable AND, my tool and all the tools I include in the process are adamantly free, and there will always be a free version of my software... eventually I'll monetize the project when I can engineer all of the tools myself, but by that point, the problems I'm looking to solve may already be solved by other companies solutions and despite what anyone may think when I say this? That's not a bad thing. That's actually a really good thing.

It's a tall order, but it's not really the entirety of what I seek to build, many of the things I want to incorporate require manipulation of service configurations, DCOM settings and permissions, File and Network permissions, Firewall entries and monitoring, Active Directory assignments and Desired State Configuration templates, automatic script elevations and automatic module integration... in entirety, that's what Hybrid | Desired State Controller seeks to be... All of it and the kitchen sink... but instead of copying everything? It just creates a blueprint or a mask of everything, and this process is called 'vectorization' – some may even refer to some of these concepts as 'containerization'.

The best analogy I can provide is this... I don't want to speak for everyone here, but I gather that most people have heard of Adobe Photoshop. With it, you can use rasterized images that require a lot more data, because it's raw and there aren't any patterns, it's just a static image and it's meant to be. It's harder to animate or move that data because it takes up a lot more space than a vectorized image. With a vectorized image, you're getting a lot more flexibility and manageability, because everything in memory is a working equation or variable... It can produce the same end result as a rasterized image can, but it does so with a lot less data. That's what vectorization means, using math and algorithms and/or codecs to save resources like time, money, processing power, hard drive space, network bandwidth, and etc.

All this is so far, is a working concept, and it will utilize the PSD-Master project located at www.github.com/FriendsOfMDT … which is also a working concept, nothing here is totally complete, but the process you see below will explain what is being developed. I've already begun to re-script the modules that they have written after asking about why the project went dead a couple of years ago, and they have relaunched the project to a publicly accessible/amendable state. The best part is that it's open source and it's free. It's also exceptionally ambitious.

These app developers and engineers need a helping hand, not overwhelming negative criticism... they write a lot of redundant code, and I'd like to cut down on that because that's how mistakes and vulnerabilities surface elsewhere. If you scope out the scripts I have already rewritten to some degree? I've found ways to perform the same activities they've put together with less code... and I'm sure that it burns some people that I've only been programming with PowerShell for 6 months... but I can tell you... I've managed to offset my lack of programming experience by seeing what programs do over the last few decades, it also helps that I know how to make graphics and design web pages.

All things considered... My intentions are purely well meaning... even if I may appear to be MS's biggest critic.Here is a link to my more recent video although an update will soon be pending...

https://www.youtube.com/watch?v=C8NYaaqJAlI
The sound is quiet until about 2 minutes in, and this is a detailed examination and explanation of the process you see..

https://github.com/secure-digits-plus-llc/Hybrid-DesiredStateController
This is my company GitHub project that people may want to investigate or scope out.

This is a list of software and tools that I use...

DISM++ Probably the most powerful utility
https://www.chuyu.me/en/index.html – Based on CBS/DCOM

Microsoft Deployment Toolkit | SCCM | ConfigManager
https://docs.microsoft.com/en-us/sccm/mdt/ - Based on WinPE

OPNSense
https://opnsense.org/ - Based on FreeBSD

VMWare Workstation Pro & Player ( former not in this video, it's on another server )
https://vmware.com

Snappy Driver Installer
https://sdi-tool.org/ - Based on SamLab and/or Microsoft Update Catalog or Hardware Vendors

Chocolatey ( Only optional, I've developed a way around needing it but will credit the inspiration )
https://chocolatey.org/

Ninite ( I no longer use this tool, as it uses Chocolatey or some other form of Powershell Gallery Utilities, and costs money for Commercial use, I have used extensively in the past when working for a company that used it )
https://ninite.org

People I would like to thank:

Michael E. Cook ( Father, deceased )
Paul Allen ( Inspiration figure, deceased )
Gary Kildall ( Inspiration figure, deceased )

Michael Niehaus ( Microsoft )
Johan Arwidmark ( DeploymentBunny )
Michael Nystrom ( DeploymentBunny )
Damien Van Robaeys ( Systandeploy )
Gary Ewan Park ( Chocolatey )
Shane Young ( Bold Zebra's )

Gary McQueen ( former coworker at KeyCorp )
Joshua Greenwood ( also KeyCorp )
Ryan Hindes ( also KeyCorp )
Matthew Roerig ( formerly Computer Answers, Nfrastructure )
Chamren Beavers (alumni / High School / CRCATS )
David Patzarian ( former CCNA/CompTIA instructor – High School / CRCATS )
Tim Smolyn ( Former CompTIA/MCP Instructor – New Horizons )
Bruce Cheney ( Former CompTIA/MCP Instructor – New Horizons )
Lara Rivenberg ( Former CompTIA/MCP Instructor – New Horizons )
Michael and Alexander Philipsak ( genesis of my desire to pursue IT / Price Chopper / Not totally certain )
Dwayne Coonradt ( mentor/advocate – Computer Answers of NY LLC )
Pavel Zaichenko ( business partner/advocate – Computer Answers of NY LLC )

Other mentions...

https://stackoverflow.com/users/3080908/lee-dailey ( Stack Exchange )
https://stackoverflow.com/users/8523330/rokumaru ( Stack Exchange )

Linus Sebastian ( Linus Tech Tips )
Louis Rossman ( Rossman Group )

There are other people I would like to thank, but have chosen to keep their names private, they know who they are...

This article or post is the first step toward publicizing my project and where I intend to take it. Any co-developers would be appreciated, however, I've learned a lot over the last 6 months of developing this process myself and with helpful community resources and the references / mentors I've listed...