On deployment and enviromental risk
One hot topic that you often get when you have project teams delivering functionality for user acceptance testing while support are testing ad hoc fixes for production issues is "Who owns the environment and how do we stop support and project work from interfering with each other?".
I feel that all code changes and deployments to customer facing sites should have some kind of change management paperwork raised. Because of this you are able to catch scheduling/work conflicts in the change advisory board. A good change control process should help you think about the risk to your project and also help flag to other stakeholders that they might have a risk to consider.
A very basic rule of thumb for deploying anything to production in a manually controlled environment is don't expect that no other changes to have been made since you took your base to work on and always check before merging in. Here I think a "manually controlled environment" is somewhere that developers and application support have write access directly to production and UAT servers and that deployment is typically done manually by a developer or application support developer copying files and installing etc.
In this environment I would consider it a project risk that any UAT sites have the potential to have emergency support changes made and/or in test. A Few things that I would do to help mitigate this in the short term:
• Use the CAB process as fully as possible, this makes everyone aware of what you intend to change and when
• Confirming with support before proceeding and referencing the CAB form – do you need to do anything? Is there any reason we shouldn’t proceed?
• Check what is there as you deploy – is it what was there before we started development? Have any CABs been raised for these clients/environments that might impact our project?
This is assuming that any inter-project team conflicts over environments can be resolved in regular PM/programme management meetings - I would place the onus on the project work as I would have thought that project deployments should be able to be planned far enough in advance to do this. IMHO it is typically a lot easier for project teams to check and plan for this than it is for support given the nature of the work.
In the longer term it makes sense to migrate those clients to an auto-deployment structure from a "source of truth", for which I would strong recommend a good source control system. Then place stricter controls on environment access at the server level. Ideally I wouldn't want anyone other than operations staff performing anything other than infrastructure tasks directly on the server. For things like logging there are enough ways of piping logs off and examining Windows event logs remotely that developers and application support shouldn't need much - if any - server access.
I feel that all code changes and deployments to customer facing sites should have some kind of change management paperwork raised. Because of this you are able to catch scheduling/work conflicts in the change advisory board. A good change control process should help you think about the risk to your project and also help flag to other stakeholders that they might have a risk to consider.
A very basic rule of thumb for deploying anything to production in a manually controlled environment is don't expect that no other changes to have been made since you took your base to work on and always check before merging in. Here I think a "manually controlled environment" is somewhere that developers and application support have write access directly to production and UAT servers and that deployment is typically done manually by a developer or application support developer copying files and installing etc.
In this environment I would consider it a project risk that any UAT sites have the potential to have emergency support changes made and/or in test. A Few things that I would do to help mitigate this in the short term:
• Use the CAB process as fully as possible, this makes everyone aware of what you intend to change and when
• Confirming with support before proceeding and referencing the CAB form – do you need to do anything? Is there any reason we shouldn’t proceed?
• Check what is there as you deploy – is it what was there before we started development? Have any CABs been raised for these clients/environments that might impact our project?
This is assuming that any inter-project team conflicts over environments can be resolved in regular PM/programme management meetings - I would place the onus on the project work as I would have thought that project deployments should be able to be planned far enough in advance to do this. IMHO it is typically a lot easier for project teams to check and plan for this than it is for support given the nature of the work.
In the longer term it makes sense to migrate those clients to an auto-deployment structure from a "source of truth", for which I would strong recommend a good source control system. Then place stricter controls on environment access at the server level. Ideally I wouldn't want anyone other than operations staff performing anything other than infrastructure tasks directly on the server. For things like logging there are enough ways of piping logs off and examining Windows event logs remotely that developers and application support shouldn't need much - if any - server access.
Comments
Post a Comment