We’ve all been there. A lot. Before your next argument about a project estimate starts, take a minute to get everyone on the same page. Estimates are in important part of business AND estimates are often wrong. Yes, both of these truths can exist in the same universe!

My favorite insight from this set of articles was from John Cutler, “What job are you hiring estimation to do?” This perspective is perhaps the most valuable thing to keep in mind as your team works through your next estimate.

• This article does a deeper dive into approaches that can be used (phases of estimates, methods, etc.) and goes much deeper than smaller shops typically need to, but it should give a perspective into the world of those who really need this.
  https://www.smartsheet.com/ultimate-guide-project-cost-estimating
• Quick and dirty reference for more accuracy in estimates: https://www.projectsmart.co.uk/12-tips-for-accurate-project-estimating.php
• An excellent article, complete with background on agile, knowledge workers, that explains what the real intent, pitfalls and advantages of estimation are. If you read nothing else, read this!
  https://www.pmi.org/learning/library/mindset-estimating-planning-agile-9340
• This one is a bit longer, but an easy read. It covers why we estimate and how to estimate and a lot of that is ground that we’ve covered already but it was a good read in regards of getting a complete picture.
  https://info.thoughtworks.com/rs/thoughtworks2/images/twebook-perspectives-estimation_1.pdf
• A sensible argument against #NoEstimates that leads to valuing what the estimate is being “hired” to do.
  https://hackernoon.com/why-do-we-estimate-6839ba295e9e

There’s some debate still about these 20x developers out there, complete geniuses that gave rise to the “rock-star” moniker that every company seemed to advertise for. I’ve remained a touch skeptical–not that there aren’t amazing software engineers, I know several — but I’m skeptical because we still don’t have a really good, accessible ways to measure human contribution to software. Lines of code, commits and story point velocity are readily available. Regressions, debugging and runtime numbers can be gathered as well, but we know that all this still doesn’t paint the whole picture. Also add in the real-world impact that other factors outside of development have on performance as well: good product management, architecture and QA among them, but also deadlines, poor communication and bad management. With all of these variables, it’s going to be pretty tough to accurate gauge order-of-magnitude differences between programmers in the real world. And frankly, I don’t know that you need to because in your gut you already know.

“The differences arising from individuals in any given study will drown out any differences you might want to attribute to a change in methodology.” — Steve McConnell

Back in 2011 Steve McConnell responded to criticisms of his 10x programmer claim thoughtfully by pointing out what he was seeing in the studies that have already been done, most not even focused on individual contribution. What he found was that “the differences arising from individuals in any given study will drown out any differences you might want to attribute to a change in methodology.” In short, it nearly totally depends on who’s on your team. However, over-emphasis on this leads to ignoring methodology and tooling altogether and focusing instead on individual development and hiring, which would be short-sighted. My point here is that on the balance, the latter should constitute a more significant portion of team building efforts.

The trouble arises when you commoditize development. While there are many software tasks that require no planning or design, there is much more that requires careful thought. As Yevgeniy Brikman said in support of the 10x developer, “It’s not about writing more code; it’s about writing the right code. You become a 10x programmer not by doing an order of magnitude more work, but by making better decisions an order of magnitude more often.”

“It’s not about writing more code; it’s about writing the right code. You become a 10x programmer not by doing an order of magnitude more work, but by making better decisions an order of magnitude more often.” — Yevgeniy Brikman

This is why I prefer the title software engineer. Writing good applications is really about a mind well-suited for building: choices, compromises and the collective wisdom of personal experience and industry best practices. Don’t commoditize development. It’s a highly-talented and highly-compensated profession. It’s not about nuturing a team, it’s about providing the humans working around you with the best possible resources to be successful, innovative and frankly, fun to be around. Yes, tools matter, and methodology matters–they matter a lot and they can have a big impact on the quality and speed of your work, but investing in these while sacrificing investment in the professional development of the individuals on the team… well… that’s as bad as it sounds.

Today I was reviewing the Moab documentation for an upcoming training and I ran across several feature gems that I thought were worth calling out. I’ll call them some of the “not-so-secret ingredients” that makes Moab great.

Scheduling with Partitions
Moab uses partitions to logical divide available resources in your environment. This allows you to separate geographically or by hardware configuration. For example, because a given job can only use resources within a single partition, you may want to create a partition to group nodes on the same local switch so that you can guarantee the fastest interprocess communication speed. Another cool benefit of partitions is the ability to set partition-specific policies, limits, priorities and scheduling algorithms, though this is rarely necessary.

Green Computing
In an effort to conserve power, Moab can automatically turn off idle nodes until they are needed again. This can be a significant savings if you’re not maxing your capacity all the time. To save on time, Moab can keep a pool of nodes on standby so that there is no delay when additional resources are needed. Moab comes with reference scripts for IPMI, but can be configured to work with iLO, DRAC and xCAT as well.

Fairshare
Using configured targets, you can use historical resource utilization as a factor for job priority. In essence, this would prevent a user with relatively infrequent workload to get buried in the backlog of jobs from much busier users. This kind of thinking extends to groups, accounts and other groupings as well. There are a number of configuration settings for fairshare that allow you to tweak performance to your needs, such as caps, evaluation timeframe, and consumption metrics.

Reservation Groups
You can associate multiple reservations through reservation groups. This allows each reservation to share the same variables. These variables can then be used in job triggers to automate tasks like email notification, script execution and internal Moab actions, such as creating another reservation. Of course, you can always override the inheritance by setting a variable locally on an individual reservation in the group.

If you’re reading this wondering why we chose to do one feature over another, please note that you’re not the first (or the last) to shake your fist while shouting this question! We hope that all of our customers are this passionate about our software. I know I am with the products I use, so we expect nothing less!

Whether VMware, HP’s Cloud Service Automation, xCAT or any of the other myriad provisioning solutions, there are so many progressive stages of private cloud. Although that term remains somewhat nebulous, the pieces are familiar:

1. Standardization and consolidation of hardware and infrastructure
2. Virtualization and automation (most of us are around here)
3. Self-service infrastructure (next step)
4. Service lifecycle management
5. Service brokering and hybrid environments

The barriers are also ever present, among them manpower, optimization, guaranteed SLA enforcement and accounting – each increasing the difficulty of making progress or really even getting a good understanding of the end-game for your private cloud. The best place to start your push to cloud is a strong focus on return on investment. Gathering an accurate understanding of current costs and demand is a significant first step, and from there a hope can build around the potential for cost savings. It is no stretch to claim between 10 and 100 times faster deployments, depending on your current setup! Our research has shown that our customers are spending 2-3 times more in manpower and hardware before moving to a cloud.

Another strong selling point for cloud and a potential for savings is the concept of a self-service portal. We’ve all dreamed of facilitating users, in a safe way, so that then can request and manage their own workload without requiring much IT team. Another bonus is the addition of chargeback concepts to help manage resources in an accountable manner. Workload placement and migration is another level of management that is expensive and time-consuming to handle manually. Even setting up the rule sets and auditing policies can be overwhelming.

So if you are focusing on a consolidation of hardware into a single datacenter, consolidating deployment efforts and processes, looking to increase your ROI on your infrastructure, decreasing IT staffing and computing investments or just wanting to add additional machines and VMs without staffing up, Adaptive Computing’s line of cloud solutions can help your success.

Self-service Portal
Using Moab’s service templates, actual service consumers can ask for what they need, when they need it. With chargeback you can govern accountability for resource uses, which will limit overuse and waste. Remember that a free cloud is a recipe for failure. Self service is what makes a your datacenter a cloud, and can facilitate 10-100x increases in deployment speed.

Continuous Optimization
Move from automation to orchestration. VM sprawl, just like server sprawl, can fill up your datacenter quickly! Do you want your VMs to be scattered across your datacenter for better performance, or would you like to consolidate so that you can take advantage of licensing constraints or VLANs. Don’t overlook the value of accurate initial placement with granular service allocation policies that can target by processors, memory, chipset, software licenses or other arbitrary metadata. Also, you can always reserve pockets of your datacenter for certain kinds of work, or work from certain users. Once those services are running, they can be locked down on that hypervisor to secure high availability and security. Now all this can be done manually, but let’s be honest, it’s a lot of work, and your ability to keep in sync with upcoming needs will be limited by staffing and the other fires and stuff your IT staff it doing. You don’t want to just add capacity like we have in the past. Now is the time to efficiently allocate what we already have with a variety of policies around placement, overcommit and allocation.

Integrating Moab in Your Cloud
Dip your toe, try this out. We know that you may want to keep one foot in the traditional IT infrastructure model, or even outsourced IaaS . We know that this perpetuates inconsistent development environments, disparate architectures and different management and security, so pick a single small group to focus on. You want to provide all the capabilities (optimization, chargeback, service catalog, etc.) for a each grouping one at a time so that you can demonstrate the ROI as you work to cloudify (bring standardization, automation and self-service) your datacenter.

Software development companies have long been abuzz with talk of agile development. Its virtues have been extolled and significant gains in productivity promised, but not very often do we talk about how hard it is to make the transition to agile development. There are compromises that can come so easily when we face processes and pressures that are difficult to transition away from. Old habits are so welcoming and progress can seem so far away. At Adaptive we experienced the same growing pains. Over the past year and a half we have seen a lot of success from our efforts and it hasn’t been easy. But it’s been worth it. We’ve been able to set more accurate timelines, and the accuracy of those timelines has been available sooner. Each release we have been able to move ahead with confidence that we can scope out the relevant functionality completely with user stories. We’ve been able to target our hiring to bring on the specialists we know that we need to have success. And we’ve been able to plan future features more fairly, having a better understanding of the customer needs we’re addressing and how long it will take to deliver value in each feature.

How does this translate to our product? We’re seeing higher quality, faster performance, longer uptimes, more scalability and more fully rounded-out features. Higher quality comes as feature is subjected to longer durations of targeting testing because we spend less of our release cycle in a no-man’s land between completed features. Performance, uptime and scalability are the result of deeper dives into specific user stories and their impact on the system and its components. Each of our new features is more fully thought out and executed as we have taken more time to examine mainstream and edge customer use cases.

In the end, we’re confident that the investment we’ve made in improving our agile development process has produced the most valuable version of Moab we’ve released, and we expect even better versions in the future!