I updated my remove all children code to make it less error-prone.  I was having an issue with the children being deleted before the for loop completed, which left some behind.  Using LINQ I was able to adjust it to make a copy of the list of children before I called destroy:

    public static void DestroyAllChildren(Transform parent)
    {
        if (parent!=null)
        {
            List<Transform> l = parent.Cast<Transform>().ToList();
            foreach (Transform c in l)
            {
                if (c!=null && c.gameObject!=null)
                {
                    if (c.childCount>0)
                        DestroyAllChildren(c);
                    Destroy(c.gameObject);
                }
            }
            parent.DetachChildren();
        }
    }

Calling parent.DetachChildren() removes the children from the parent immediately.