The most productive of evenings

It has been an extremely productive evening, both on the code front and on the DevOps front.

Code:

I had a bug where I was clearing a div and then attempting to repopulate it.  For the life of me, I could not get it to populate any HTML after calling jQuery's empty() function.

    

    function removeAllSections() {
        var agendaId = $('#AgendaId').val();
        $.post("/Agenda/SectionDeleteAll",
            {
                "agendaId": agendaId
            },
            function () {
                var agendaDiv = $("[name='agenda-nestable']");
                agendaDiv.empty();
            }, "json");
    }

Oh, haha, silly me. The div contains an ol, and I was clearing the contents of the div, and then attempting to add list elements to an ol that did not exist. Fixed:

    

    function removeAllSections() {
        var agendaId = $('#AgendaId').val();
        $.post("/Agenda/SectionDeleteAll",
            {
                "agendaId": agendaId
            },
            function () {
                var agendaRoot = $("[name='agenda-nestable'] ol#agenda-root");
                agendaRoot.empty();
            }, "json");
    }

On the DevOps front, I was wondering why my Razor Views take so long to load, even with Azure "Always On" enabled.

Well, tonight I found this beauty:

(then click Configure link)

Boom, precompiled Razor CSHTML views stored in their own Managed Assembly.  

Things are running much faster now!!!  No more initial page load taking forever (while the .cshtml compiles).  

Govern CMS EDMX Diagram

I'm making good progress on my new Government Content Management System (Govern CMS).

Probably have another 1-2 weeks of development before I have a minimum viable product.

Here is my Entity Framework EDMX for that System:

HTML Helpers - ASP.NET MVC's equivalent of JSTL Custom Taglibs

I have found the equivalent of Java's JSTL Custom Taglibs: HTML Helpers.

I started moving logic out of my Razor Views and into HTML Helper classes.

Here's my first, and it's a beauty (uses recursion to build out nested lists):

    public static class CategoryExtensions
    {
        public static IHtmlString CategoryDisplay(this HtmlHelper helper, IEnumerable categories, int indentSize)
        {
            string indentString = "";
            for (int i = 0; i < indentSize; i++)
            {
                indentString += " ";
            }
            StringBuilder stringBuilder = new StringBuilder();
            stringBuilder.Append("
\n");
            stringBuilder.Append(BuildList(categories, indentString));
            stringBuilder.Append(indentString).Append("
");
            return new HtmlString(stringBuilder.ToString());
        }

        private static IHtmlString BuildList(IEnumerable categories, string indentString)
        {
            StringBuilder stringBuilder = new StringBuilder();

            stringBuilder.Append(indentString).Append("  
\n");

            foreach (Category category in categories)
            {
                stringBuilder.Append(indentString).Append("    
\n");
                stringBuilder.Append(indentString).Append("      
\n");
                stringBuilder.Append(indentString).Append("        " + category.CategoryName + "\n");
                stringBuilder.Append(indentString).Append("      
\n");
                if (category.SubCategories.Any())
                {
                    indentString += "  ";
                    stringBuilder.Append(BuildList(category.SubCategories, indentString));
                }
                stringBuilder.Append(indentString).Append("    
\n");
            }
            stringBuilder.Append(indentString).Append("  
\n");

            return new HtmlString(stringBuilder.ToString());
        }
    }

I'm glad to get the logic out of the Razor views (which should not have such complex logic) and into proper C# classes.

Angular 4 node_modules

Wow ... I'm planning on building my Angular 4 App this weekend.  I am working through the tutorial and just had a moment of horror ...

217 MB for a JavaScript Framework?!  You've got to be kidding me.  

There are over 700 subdirectories under node_modules. 

Now, I did take a look at the HTML generated and it does not appear to reference node_modules.  I am going to see if I can get away with NOT deploying it.

Angular 4

I'm starting on Angular 4. Going through the Tour of Heroes Tutorial.

I've done plenty of Angular 1.x in industry and feel comfortable with it.

Angular 4 looks like a completely different beast altogether.

Some initial thoughts:

• Coming from a C#/Java background, I like TypeScript
• However, I do not like the concept of transpiling. I think it's asking for trouble, and I have no doubt at some point I will end up troubleshooting/debugging ugly transpiled JavaScript.  
• I really think that the JavaScript V8 Engine and other JavaScript Engines should run bytecode.  Both JavaScript and TypeScript should be compiled 1st class languages, similar to how Java and Groovy are both compiled to ByteCode, as C# and F# are both compiled to MSIL.
• I've found my IDE: Visual Studio Code. This thing is beautiful and responds very quickly and feels light and easy to use.  It's even better than WebStorm and Visual Studio .NET.  I do hate having 2 IDEs (Visual Studio Code, Visual Studio .NET) for front-end/back-end though. 

Govern CMS

To-date, I've filled out 45 RFPs.

One common trend I've noticed is a need for Content Management System (CMS) Software.

Other than Website Redesign (which cannot be generalized), CMS comes in 2nd for "Most Requested Software".

So, I figured I would build a CMS System.  I have an informal CMS backing Municipal Agenda, so I figured i'd take the time to extract and formalize it into a standalone product.

To get there ...

First, the Data Model.  Everything starts with the Data Model.

More to come.

I've thought about using this as my opportunity to learn F#. 

The problem I will encounter is that this is not a common language, and if customers want to look under the hood, it will be a mark against me.  Nope, going with the tried and true C#. 

WCAG 2.0 Color Contrast Ratios

Today I learned about WCAG 2.0 Color Contrast Ratio Requirements.

WCAG 2.0 requires that the foreground and background colors have a 4.5:1 contrast ratio at Level AA and a 7:1 contrast ratio at Level AAA.

Now, I don't know what a 7:1 contrast ratio looks like, but my gut feel is it's like black on white or white on black.

So, I'm targeting WCAG 2.0 AA Compliance for all my sites.  That's sort of the gold standard, anyway. 

My WCAG-compliant labels: 😊

.label-default {
    color: #FFF;
    background-color: #777;
}
.label-danger {
    color: #FFF;
    background-color: #a30909;
}
.label-warning {
    color: #FFF;
    background-color: #a36c06;
}
.label-success {
    color: #FFF;
    background-color: #124400;
}
.label-info {
    color: #FFF;
    background-color: #027091;
}
.label-primary {
    color: #FFF;
    background-color: #003FBB;
}

Twitter Bootstrap 4

I've migrated my Recruiter Review Site to Twitter Bootstrap 4 so that I could use a more modern Bootstrap Theme: Real BS 4.

To say that it's been painful would be generous.

Most of my pain has been caused by Twitter's decision to remove Glyphicons from Bootstrap 4.

I would love to know why anyone thought that this was a good idea.

It broke my Bootstrap Star Rating widget, and it's been a LONG road back this evening, after I downloaded the source code for the widget and took it apart, turned it upside-down before figuring out the cause.

For anyone who needs to re-introduce Glyphicons to Bootstrap 4, follow this Stackoverflow Post.

HTTPS everywhere

I am enforcing HTTPS everywhere on my new projects.

In order to do so, I redirect any attempt to access HTTP URLs to HTTPS. 

This was a bit of a struggle.

I tried the recommended approach:

This did not work, which was surprising. I ended up having to install an Azure Web App Extension:

Now, I am forcing HTTPS everywhere, on all my resources.

WCAG 2.0 Compliance

So, a lot of the RFPs I've been bidding on are looking for WCAG 2.0 Compliance (accessibility for people with disabilities).

I figured the best thing to do is make my own site WCAG 2.0 Compliant.

I am striving for AA status on all 4 categories.

So far, here is what I am finding:

• Color Contrast.  This is big.  Like no light yellow buttons with white text.  Okay, that's atrocious for fully-abled people, but still, I've had to make a lot of changes along those lines. The biggest net effect is my site looks cleaner, crisper, more defined.  There were some beautiful aquas that had to go, since neither black nor white text provided enough contrast.  I also made my help blocks solid, instead of a faded out ghostly grey.  All of these are overall site aesthetic improvements. 
• alt attribute for img tags.  This is just good HTML practice.  I used to do that consistently as a high school kid building websites, I just got lazy.
• aria-label attributes for input tags.  This is new to me, never heard of that attribute before today. Looks like it's an assistive technology thing.

HTML5 Video hosted in Azure Storage

If you are like me and hosting Video files (MPEG / WEBM / OGG) in Azure Storage, you will probably need to do the following so that the MIME type comes down correctly:

        public static CloudBlockBlob UploadAndSaveBlob(CloudBlobContainer blobContainer, String text, String contentType, String extension)
        {
            string blobName = Guid.NewGuid().ToString() + extension;
            // Retrieve reference to a blob. 
            CloudBlockBlob blob = blobContainer.GetBlockBlobReference(blobName);
            // Create the blob by uploading a local file.
            blob.UploadText(text);
            // Set the Blob content type
            blob.Properties.ContentType = contentType;
            blob.SetProperties();

            logger.Info(String.Format("Uploaded text file to {0}", blob.Uri));

            return blob;
        }

and call the method, passing in a contentType of "video/mp4" for example.

This will set a ContentType Property on your Blob.

Also, you will probably need to install a WebM plugin for IE (which is oddly enough put out by Google, nice to see they're looking out for IE).

Back on another project and new EDMX

Finally settled on a 3rd Party Mail Sender: SendGrid.

Out of all the ones I tried:

• SendGrid
• MailGun
• SparkPost

SendGrid is the only one that has consistently sent out my emails.  Which is an important feature of a 3rd party emailer.

Plus, Azure provides free sending of SendGrid Emails with your subscription. 

If Azure provided SMTP servers and a mail infrastructure, I totally would have just handled this in-house.  However, they outsourced this to SendGrid.

My new EDMX.  Still elegant, but growing:

The magic of LINQ

LINQ is one of my favorite features of the Microsoft .NET Framework.

 It was introduced with VS.NET 2008 / .NET 3.5, and is absolutely incredible.

I often wish we had something like it in Java, where I'm still cobbling together foreach loops to process Object Lists, SAX Parsers to process XML, and HQL/JP-QL to select data. I needed to take a list of Objects and write out XML Export in .NET.

 Here's my code:

            XDocument xmlDocument = new XDocument(new XElement("Agendas",
                from agenda in orgAgendas
                select new XElement("Agenda",
                    new XAttribute("Id", agenda.AgendaId),
                    new XElement("AgendaName", agenda.AgendaName),
                    new XElement("MeetingDateTime", agenda.MeetingDateTime),
                    new XElement("MeetingLocation", agenda.MeetingLocation),
                    new XElement("OriginalFileUrl", agenda.AgendaOrigUrl),
                    new XElement("ConvertedFileUrl", agenda.AgendaPdfUrl),
                    new XElement("Type", agenda.Type),
                    new XElement("TypeDesc", agenda.TypeDesc),
                    new XElement("CreateDate", agenda.CreateDate),
                    new XElement("Published", agenda.Publish),
                    new XElement("Sections",
                    from section in agenda.AgendaSections
                    select new XElement("Section", 
                        new XAttribute("Id", section.SectionId),
                        new XElement("Text", section.SectionText),
                        new XElement("Type", section.Type),
                        new XElement("Owner", userUtils.GetUserNameById(section.OwnerId.GetValueOrDefault())),
                        new XElement("Items",
                        from item in section.AgendaSectionItems
                            select new XElement("Item", 
                            new XAttribute("Id", item.ItemId),
                            new XElement("Text", item.ItemText),
                            new XElement("Owner", userUtils.GetUserNameById(item.OwnerId.GetValueOrDefault())),
                            new XElement("Assignee", userUtils.GetUserNameById(item.AssigneeId.GetValueOrDefault())),
                            new XElement("Completed", item.Completed),
                            new XElement("DueDate", item.DueDate),
                            new XElement("Approved", item.Approved),
                            new XElement("Attachments",
                            from attachment in item.Attachments
                                select new XElement("Attachment",
                                new XAttribute("Id", attachment.AttachmentId),
                                new XElement("Description", attachment.Description),
                                new XElement("OriginalFileUrl", attachment.OriginalUrl),
                                new XElement("ConvertedFileUrl", attachment.ConvertedUrl),
                                new XElement("Confidential", attachment.Confidential)
                                )   
                            )
                            )
                        )
                    )
                    )
                )
            ));

So elegant ... :-)

Entity Framework Woes

I started getting a new and horrifying Entity Framework error I had never seen before. Conflicting changes detected. This may happen when trying to insert multiple entities with the same key. It looked like it was due to me saving a parent and having it cascade to save children. Soooo, I took it upon myself to save parent, save child, reconstitute relationship. This was horrifying. My code went from Exhibit A:

        /// 

        /// Helper Method.  Updates Sections and Items
        /// 
        /// Agenda ID
        /// List of AgendaSections
        /// The Current User creating Sections and Items
        private void UpdateSectionsAndItems(int agendaId, IList<AgendaSection> agendaSections, IUser currentUser)
        {
            List<AgendaSectionItem> agendaSectionItems = new List<AgendaSectionItem>();

            int sectionNumber = 0;
            foreach (AgendaSection section in agendaSections)
            {
                section.Number = sectionNumber;
                section.AgendaId = agendaId;
                section.OwnerId = currentUser.UserId;
                if (section.AgendaSectionItems != null)
                {
                    int itemNumber = 0;
                    foreach (AgendaSectionItem item in section.AgendaSectionItems)
                    {
                        item.Number = itemNumber;
                        itemNumber++;
                        item.OwnerId = currentUser.UserId;
                        agendaSectionItems.AddRange(section.AgendaSectionItems);
                    }
                }
                sectionNumber++;
            }
            db.AgendaSections.AddRange(agendaSections);
            db.AgendaSectionItems.AddRange(agendaSectionItems);
            db.SaveChanges();
        }

To the following horrifying monstrosity:

        /// 

        /// Helper Method.  Updates Sections and Items
        /// 
        /// Agenda ID
        /// List of AgendaSections
        /// The current user
        private void UpdateSectionsAndItems(int agendaId, IList<AgendaSection> agendaSections, IUser currentUser)
        {            
            // This is a horrible workaround to deal with this mysterious error: 
            //   Conflicting changes detected. This may happen when trying to insert multiple entities with the same key.
            // That I get when I simply try to save parent (AgendaSection) and let it cascade down to the child (AgendaSectionItems).
            IDictionary<int, List<AgendaSectionItem>> tempItemDictionary = new Dictionary<int, List<AgendaSectionItem>>();
            int sectionNumber = 0;
            foreach (AgendaSection section in agendaSections)
            {
                section.Number = sectionNumber;
                section.AgendaId = agendaId;
                if (section.AgendaSectionItems != null)
                {
                    int itemNumber = 0;
                    foreach (AgendaSectionItem item in section.AgendaSectionItems)
                    {
                        item.Number = itemNumber;
                        itemNumber++;
                        item.OwnerId = currentUser.UserId;
                        if (tempItemDictionary.ContainsKey(section.Number.GetValueOrDefault()))
                        {
                            List<AgendaSectionItem> agendaSectionItems = tempItemDictionary[section.Number.GetValueOrDefault()];
                            agendaSectionItems.Add(item);
                        }
                        else
                        { 
                            List<AgendaSectionItem> agendaSectionItems = new List<AgendaSectionItem>();
                            agendaSectionItems.Add(item);
                            tempItemDictionary.Add(section.Number.GetValueOrDefault(), agendaSectionItems);
                        }
                    }
                }
                // More workaround
                section.AgendaSectionItems = null;
                sectionNumber++;
            }
            db.AgendaSections.AddRange(agendaSections);
            db.SaveChanges();

            // More workaround ... FML
            foreach (int i in tempItemDictionary.Keys)
            {
                IList<AgendaSectionItem> items = tempItemDictionary[i];
                AgendaSection agendaSection = agendaSections.First(s => s.Number == i);
                if (agendaSection != null)
                {
                    foreach (AgendaSectionItem item in items)
                    {
                        item.SectionId = agendaSection.SectionId;
                        if (agendaSection.AgendaSectionItems == null)
                        {
                            agendaSection.AgendaSectionItems = new List<AgendaSectionItem>();
                        }
                        agendaSection.AgendaSectionItems.Add(item);
                        db.AgendaSectionItems.Add(item);
                        db.SaveChanges();
                    }
                }
            }            
        }

When I finally got down to it, what I had done was accidentally mapped my primary key on a table as a foreign key in a completely incorrect mapping. So, I went and rolled back the monstrosity and will sleep better tonight.

Why can't I find a 3rd party product to send emails that works?

*sigh* it should not be that difficult.

So far, I have tried:

• SendGrid:  They seemed to work.  However, I inadvertently spun up a new project and committed my SendGrid API Key in my source code to GitHub.  Apparently they like being Big Brother and scan GitHub for their API Keys and shut down accounts when they find it.  Great.  Shut down my account, that's fine I'm moving on ...
• MailGun: This did ... not ... work.  I walked it in the debugger, no errors, went through all their documentation.  Just did not work. I don't have the time, patience, or bandwidth to make it work.
• SparkPost: My current attempt.  Really hoping 3rd time is a charm. 

Introducing: new EDMX and a personal challenge

I am taking a break to build a Recruiter Review Website.

The Data Model is much simpler:

I am a firm believer that a good Senior Developer should be able to go into a 1 hour Interview and build a fully functional website.

I am challenging myself to build this new site in 1 day.  A 1 hour Website should be functional, but will not look polished and have nice widgets and UI, but I really do believe a good Senior Developer should be able to build a functional CRUD site in an hour.

Introducing: my EDMX Diagram

I am building my own Software Project.

Here is my Data Model (Entity Framework EDMX Diagram).

You know ... that horrifying moment when you ask yourself "is my Data Model too complex?"

Seriously though, I went with the simplest Data Model possible given the scope and functionality of my site, and keeping with 3rd Normal Form, and this is what I arrived at.

Azure WebJobs

Document Conversion Job (WebJob)

1.      Release Build from Visual Studio

2.  Navigate to DocumentConversionJob\bin\Release and zip everything into Release.zip

To Deploy open https://portal.azure.com

Azure Web App -> Web Jobs ->  Delete existing DocumentConversionJob

Add Web Job.

Screen capture of “Add Web Job” Interface:

3  3.  For checking on the health of the WebJob, there will be a URL:

https://municipalagenda.scm.azurewebsites.net/azurejobs/#/jobs

C# 4.0 - dynamic ... my new best friend

Okay, I can see the opportunity for all sorts of abuse if used incorrectly.

However, I found the perfect use for C# dynamic keyword ... dealing with JSON.

Often times, I need to send back a JSON Result with a subset of my object graph, or a flattened object graph.  I only want to send back the fields that the front end needs (trying to be judicious with bandwidth).

  dynamic result = new Object();
  result.AgendaOrigFileName = agenda.AgendaOrigFileName;
  result.AgendaId = agenda.AgendaId;
  return Json(result);

Microservices in Azure

I've started deploying Microservice VMs in the Microsoft Azure Cloud.

My VMs convert Microsoft Office Documents to PDF.  I believe this is a perfect use of Microservice / appliance Virtual Machines.

Here's how I set up the VMs.

Document Conversion VM Create/Config

1. Deploying files via copy/paste (never able to get Web Deploy working).  This was done by editing the RDP Connection->Local Resources->(check) Clipboard
2. VM -> Network Interfaces -> inbound Security Rules.  Add http / port 80
3. VM -> Network Interfaces -> Outbound Security Rules.  Add http / port 80
4. Provide a Domain Name for the VM.  Go to VM -> Properties, click on the link with IP/DNS.  Will see the following interface, enter a DNS Name.

ASP.NET MVC ViewModel not updating in View

I ran into a new and very painful ASP.NET MVC issue.

I was updating a ViewModel property in my Controller, but was not seeing the value updated in my View.

[HttpPost]
[ValidateAntiForgeryToken]
public ActionResult Manage(ManageAgenda manageAgenda)
{
    // Save an Agenda and get back a sequence-generated Unique ID
    manageAgenda.AgendaId = agenda.AgendaId;
    return View(manageAgenda);
}

On my View:

@using (Html.BeginForm("Manage", "Agenda", FormMethod.Post, new { @id = "ManageAgendaForm"}))
    {
        @Html.AntiForgeryToken()
        @Html.HiddenFor(model => model.AgendaId)
        

Now, this became REALLY painful because my ID kept coming back as 0, so upon each subsequent save, a brand new record was being created ...
The fix for this turned out to be adding the following line of code:

[HttpPost]
[ValidateAntiForgeryToken]
public ActionResult Manage(ManageAgenda manageAgenda)
{
    // Save an Agenda and get back a sequence-generated Unique ID
    manageAgenda.AgendaId = agenda.AgendaId;
    // Send back the Agenda ID.
    // Clear out the ModelState so that the ViewModel will be updated in the View.
    // See: https://stackoverflow.com/a/9645580/463196
    ModelState.Clear();
    return View(manageAgenda);
}

What scares me is I *do not understand* why this works. Other comments just described it as a bug in ASP.NET MVC. As a Software Engineer, I do not like fixes I cannot explain to others or myself.