Simple paradigm of scientific software testing

Testing of complicated scientific tasks

 

MainPicture.png

안성우 NAS/ Developer\EXAMPLE_CODE\CCSHARP\Science  참조할것
Useful links

1. Introduction

Science and engineering software can have very complicated logics. Unit testing of this software cannot test every execution path of program. This article is devoted to simple paradigm of scientific software testing. This paradigm requires saving calculation result. Testing procedure compares saved data with (new) results of calculation. If modification is correct both results should coincide. This article contains elegant code implementation of this paradigm.

2. Unit testing

Unit testing provides rapid fixing of simple bugs. I think that good testing should include both

  • Unit testing
  • Complicated testing

This section describes examples of unit testing of formula editor. Any advanced (not only) scientific software operates with formulae. Described here framework contains formula editor. Samples of formulae are presented below. Following picture contains four arithmetic operations:

These and other elementary formulae can be easy tested. Following snippet contains test of these formulae.

/// <summary>
/// Test of double binary
/// </summary>
[TestMethod]
public void TestMethodBinaryDouble()
{
    double type = 0;
    Dictionary<string, object> d = new Dictionary<string, object>()
    {
        {"x", type},
        {"y", type}
    };
    FormulaEditor.VariableDetectors.ExtendedDictionaryVariableDetector det =
        new FormulaEditor.VariableDetectors.ExtendedDictionaryVariableDetector(d);
    FormulaEditor.Interfaces.IFormulaObjectCreator cr = ExtendedFormulaCreator.GetCreator(det);
    XmlDocument doc = new XmlDocument();
    doc.LoadXml(Properties.Resources.BinaryDouble);             // Reading of formulae from XML file
    List<ObjectFormulaTree> l = new List<ObjectFormulaTree>();
    XmlNodeList nl = doc.DocumentElement.ChildNodes;
    foreach (XmlElement e in nl)
    {
        MathFormula f = MathFormula.FromString(sizes, e.OuterXml);   // Creation of formula objects
        ObjectFormulaTree tree = ObjectFormulaTree.CreateTree(f.FullTransform(null), cr);
        l.Add(tree);
    }
    Dictionary<Func<double, double, double>, object[]> dic = new Dictionary<Func<double, double, double>, object[]>()   // Dictionary of tested
    {                                                         // binary functions
        {(double x, double y) => { return x + y;} , new object[]{new object[] {"plus",  l[0]}}},
        {(double x, double y) => { return x - y;} , new object[]{new object[] {"minus",  l[1]}}},
        {(double x, double y) => { return x * y;} , new object[]{new object[] {"mult1",  l[2]}, new object[] {"mult2",  l[3]}}},
        {(double x, double y) => { return x / y;} , new object[]{new object[] {"frac",  l[4]}}},
        {Math.Pow , new object[]{new object[] {"pow",  l[5]}}},
        {Math.Atan2 , new object[]{new object[] {"atan2",  l[6]}}},
    };
    FormulaEditor.Interfaces.ITreeCollectionProxy proxy = l.ToArray().CreateProxy();   // Creation of proxy code
    foreach (Func<double, double, double> func in dic.Keys)
    {
        object[] o = dic[func];
        foreach (object[] ob in o)
        {
            ObjectFormulaTree tree = ob[1] as ObjectFormulaTree;
            string fs = ob[0] + "";
            GetValue g = proxy[tree];                                       // Proxy functions
            for (int i = 0; i < 10; i++)
            {
                double x = 0.007 + 0.07 * i;
                det["x"] = x;
                for (int j = 0; j < 10; j++)
                {
                    double y = 0.34 + 0.031 * j;
                    det["y"] = y;
                    double a = func(x, y);                                      // Calculation of functions
                    object b = tree.Result;                                     // Calculation of tree
                    Assert.AreEqual(a, b);                                      // Comparation of results
                    proxy.Update();
                    object c = g();                                             // Calculation of proxy code
                    Assert.AreEqual(a, c);                                      // Comparation of results
                }
            }
        }
    }
}

This test is very clear. It compares calculatinos obtained from formulae end C# functions.

3. Complicated testing

Unfortunately unit testing is not panacea. Unit testing cannot embrace all complicated tasks. For exampledetermination of orbits of artificial Earth satellites contains lots of elements which are presented below:
OrbitDetermination.png
Main idea of testing such use cases is recording of results. Testing contains comparation of recorded and calculated results. If calculation framework is scalable then testing should have abstract level. Following code contains this absract level.

/// <summary>
/// Test
/// </summary>
public interface ITest
{
    /// <summary>
    /// Tests collection of components
    /// </summary>
    /// <param name="collection">Collection of components</param>
    /// <returns>Test result</returns>
    object this[IComponentCollection collection]
    {
        get;
    }
}

This interface contains one method which tests collection of components and returns tests results.

3.1 Elementary tests

3.1.1 Test of time dependent function

A lot of engineering tasks contains calculation of time dependent functions. Following picture presents calculation of time dependent function
TestOfChart.png
Input signal is transformed by transfer function. Transformation result is time dependent function which is indicated by red curve on Chart component. Properties of Input are presented below:

Above picture contains formula of input signal with Dirac δ function. The Transfer function object has following properties:

It contains formula of transfer function.
Result of transformation is presented below:

Output function has jump at time= 0.5. This jump is caused by Dirac δ function.
Testing of calculation of time dependent function can be performed by following way:

  • Saving of time dependent functions;
  • Comparing of new calculation with saved result.

Following snippet contains implementation of this test.

  /// <summary>
 /// Test of time dependent function
 /// </summary>
[Serializable()]
 class LocalChart : ITest, ISerializable
 {
     #region Fields

     /// <summary>
     /// Saved series
     /// </summary>
     Dictionary<string, LocalSeries> series = new Dictionary<string, LocalSeries>();
     string name;
     string argument;
     double start;
     double step;
     int stepCount;
     string[] values = null;
     #endregion

     #region Ctor

     internal LocalChart(string name, double start, double step, int stepCount, string argument, string[] values)
     {
         this.name = name;
         this.start = start;
         this.step = step;
         this.stepCount = stepCount;
         this.argument = argument;
         this.values = values;
     }
     /// <summary>
     /// Loads saved time dependent functions
     /// </summary>
     /// <param name="info">Serialization info</param>
     /// <param name="context">Streaming context</param>
     private LocalChart(SerializationInfo info, StreamingContext context)
     {
         name = info.GetString("Name");
         series = info.GetValue("Series", typeof(Dictionary<string, LocalSeries>)) as Dictionary<string, LocalSeries>;
         argument = info.GetString("Argument");
         start = info.GetDouble("Start");
         step = info.GetDouble("Step");
         stepCount = info.GetInt32("StepCount");
     }
     #endregion

     #region ISerializable Members

     /// <summary>
     /// Saves time dependent functions
     /// </summary>
     /// <param name="info">Serialization info</param>
     /// <param name="context">Streaming context</param>
     void ISerializable.GetObjectData(SerializationInfo info, StreamingContext context)
     {
         info.AddValue("Name", name);
         info.AddValue("Series", series, typeof(Dictionary<string, LocalSeries>));
         info.AddValue("Argument", argument);
         info.AddValue("Start", start);
         info.AddValue("Step", step);
         info.AddValue("StepCount", stepCount);
     }
     #endregion

     #region ITest Members

     /// <summary>
     /// Tests collection of components
     /// </summary>
     /// <param name="collection">Collection of components</param>
     /// <returns>Test result</returns>
     object ITest.this[IComponentCollection collection]
     {
         get
         {
             IDesktop desktop = collection as IDesktop;
             IDataConsumer dataconsumer = desktop[name] as IDataConsumer;
             Dictionary<string, DataPerformer.Basic.Series> d = GetSeries(collection); // Calculation of time dependent function
             List<string> l = new List<string>();
             foreach (string s in d.Keys)
             {
                 if (!series[s].Compare(d[s])) // Comparation of test results
                 {
                     l.Add("Different series values. Object - " + name + ". Series - " + s + ".");
                 }
             }
             if (l.Count == 0)
             {
                 return null;
             }
             return l;
         }
     }
     #endregion

     #region Members

     /// <summary>
     /// Crates tests
     /// </summary>
     /// <param name="collection">Collection of components</param>
     internal void Create(IComponentCollection collection)
     {
         Dictionary<string, DataPerformer.Basic.Series> d = GetSeries(collection);
         series.Clear();
         foreach (string key in d.Keys)
         {
             series[key] = new LocalSeries(d[key]);
         }
     }
     Dictionary<string, DataPerformer.Basic.Series> GetSeries(IComponentCollection collection)
     {
         IDesktop desktop = collection as IDesktop;
         IDataConsumer dataConsumer = desktop.GetObject(name) as IDataConsumer;
         string[] ss = (values == null) ? series.Keys.ToArray() : values;
         return dataConsumer.GetSeries(start, step, stepCount,
             argument, ss);
     }
     internal string Name
     {
         get
         {
             return name;
         }
     }
     #endregion
 }

Essential part of this code is presented below

 /// <summary>
/// Tests collection of components
/// </summary>
/// <param name="collection">Collection of components</param>
/// <returns>Test result</returns>
object ITest.this[IComponentCollection collection]
{
    get
    {
        IDesktop desktop = collection as IDesktop;
        IDataConsumer dataconsumer = desktop[name] as IDataConsumer;
        Dictionary<string, DataPerformer.Basic.Series> d = GetSeries(collection); // Calculation of time dependent function
        List<string> l = new List<string>();
        foreach (string s in d.Keys)
        {
            if (!series[s].Compare(d[s])) // Comparation of test results
            {
                l.Add("Different series values. Object - " + name + ". Series - " + s + ".");
            }
        }
        if (l.Count == 0)
        {
            return null;
        }
        return l;
    }
}

Our application can be started with test support. If application is started with -tc parameter of command line (Aviation.Wpf3D.Advanced.exe -tc) then it supports tests i.e. any saving initialize request of test parameters. User interface of test request is presented below:

This picture means that current scenario is saved with test of Chart component. Saving of this scenario also saves LocalChart object
Loading of scenario also loads and performs test of Chart object.

3.1.2 Test of test

Above chapter contains tests. But we should test work test whether test works. A test without bug is not test. If we have no but we should make it. Following snippet contains artificial bug.

     //AddXY(Converter.ToDouble(x()()), Converter.ToDouble(y()()));
 /*!!! Test of test   (Artificial bug) */
AddXY(Converter.ToDouble(x()()), Converter.ToDouble(y()()) + 0.0001);
 //End test of test*/

Adding of 0.0001 is necessary calculation bug. If we load above situation with this bug then we obtain following test report:
TestReportOfChart.png

3.1.3 Test of nonlinear regression

Nonlinear regression in statistics is the problem of fitting a model.
RegressionFormula.jpg
to multidimensional xy data, where f is a nonlinear function of x, with regression parameter θ. Vector ε=(ε1,…, εn) is called vector of residuals which. Regression algorithm defines such vector θ residual parameter σ2=(ε12+…+εn2)/n is minimal. Regression algorithm iteratively estimates new value of θ. During one iteration algorithm defines new value of θ and new value of σ2. Test of nonlinear regression implies recording number of iterations and residual parameter. Following code snippet contains implementation of this test

/// <summary>
 /// Test of nonlinear regression
 /// </summary>
 [Serializable()]
 class RegressionTest : ITest, ISerializable
 {
     #region Fields

     /// <summary>
     /// Name of component on desktop
     /// </summary>
     string name;
     /// <summary>
     /// Number of iterations
     /// </summary>
     int number;
     /// <summary>
     /// Residual parameter
     /// </summary>
     double value;
     #endregion

     #region Ctor

     /// <summary>
     /// Constructor
     /// </summary>
     /// <param name="name">Name of component on desktop</param>
     /// <param name="number">Number of iterations</param>
     internal RegressionTest(string name, int number)
     {
         this.name = name;
         this.number = number;
     }
     /// <summary>
     /// Deserialization constructor
     /// </summary>
     /// <param name="info">Serialization info</param>
     /// <param name="context">Streaming context</param>
     private RegressionTest(SerializationInfo info, StreamingContext context)
     {
         name = info.GetString("Name");
         number = info.GetInt32("Number");
         value = info.GetDouble("Value");
     }
     #endregion

     #region ITest Members

     /// <summary>
     /// Tests collection of components
     /// </summary>
     /// <param name="collection">Collection of components</param>
     /// <returns>Test result</returns>
     object ITest.this[IComponentCollection collection]
     {
         get
         {
             if (GetValue(collection) != value)  // If calculated value of residual parameter is not equal
                                                     // is not equal to saved value of residual parameter
             {
                 return "Different regression values. Object - " + name;  // Then method returns error message
             }
             return null;        // Null means absence of error
         }
     }
     #endregion

     #region ISerializable Members

     /// <summary>
     /// ISerializable interface implementation
     /// </summary>
     /// <param name="info">Serialization info</param>
     /// <param name="context">Streaming context</param>
     void ISerializable.GetObjectData(SerializationInfo info, StreamingContext context)
     {
         info.AddValue("Name", name);
         info.AddValue("Number", number);
         info.AddValue("Value",value);
     }
     #endregion

     #region Members

     internal int Number
     {
         get
         {
             return number;
         }
     }
     internal string Name
     {
         get
         {
             return name;
         }
     }
     internal void Create(IComponentCollection collection)
     {
         value = GetValue(collection);
     }
     /// <summary>
     /// Calculates value of residual parameter
     /// </summary>
     /// <param name="collection">Collection of objects</param>
     /// <returns>Residual parameter</returns>
     double GetValue(IComponentCollection collection)
     {
         IDesktop desktop = collection as IDesktop;
         AliasRegression reg = desktop.GetObject(name) as AliasRegression; // Regression component
         for (int i = 0; i < number; i++)
         {
             reg.FullIterate();                  // Iteration cylce
         }
         return reg.SquareResidual;              // returns residual parameter
     }
     #endregion

 }

This class is serializable. This test object can be saved to stream. Following snippet contains essential part of above implementation

 /// <summary>
/// Tests collection of components
/// </summary>
/// <param name="collection">Collection of components</param>
/// <returns>Test result</returns>
object ITest.this[IComponentCollection collection]
{
    get
    {
        if (GetValue(collection) != value)  // If calculated value of residual parameter is not equal
                                                // is not equal to saved value of residual parameter
        {
            return "Different regression values. Object - " + name;  // Then method returns error message
        }
        return null;        // Null means absence of error
    }
}
/// <summary>
/// Calculates value of residual parameter
/// </summary>
/// <param name="collection">Collection of objects</param>
/// <returns>Residual parameter</returns>
double GetValue(IComponentCollection collection)
{
    IDesktop desktop = collection as IDesktop;
    AliasRegression reg = desktop.GetObject(name) as AliasRegression; // Regression component
    for (int i = 0; i < number; i++)
    {
        reg.FullIterate();                  // Iteration cylce
    }
    return reg.SquareResidual;              // returns residual parameter
}

In fact this test compares saved value of σ2 with calculated one.

3.1.4 Example of regression test. Determination of Orbits of Artificial Satellites

Determination of orbits of artificial satellites is considered in my previous article. Following picture contains elements of orbit determination
OrbitDetermination.png
Main component is Prosessor which performs determination of orbit.
If we try to save this scenario then obtain following request of test.

This picture has following meaning

  • The Processor component is tested
  • Count of iteration is 3

If we set to Order default value (=0) then test of processor will be dropped. Saving of this scenario also saves RegressionTest object which contains number of iterations and corresponding σ2. If we load saved file then Processor should perform 3 iterations and compares saved σ2 with calculated one. Testing of this test can be performed by following artificial bug.

th.UpdateChildrenData(); ///TEST!!! Comment this string for artificial bug of orbit determination

In case of this artificial following test report will occur.
TestReportOrbitDetermination.png

This sample requires installation of helper files. Following archive should be unpacked to directory of Aviation.Wpf3D.Advanced.exe file.

3.2 Integrated tests

Complicated engineering problems as rule contains a set of elements which need tests. In general test result depends on order of testing. For example time dependency can depend on estimated by regression parameters. In this case result of time dependency test depends on whether estimation was performed before test or after it. In this chapter some examples of integrated tests is considered.

3.2.1 Chart reconstruction

Let us consider problem of chart reconstruction from image. Sample of image is presented below:

We would like to find math function which corresponds to this image. We also suppose that image contains errors of measurements. Following picture contains elements of this task.
ChartReconstructionScenario.png
First of all we need scale this image. Moreover X axis and Y axis are not always horizontal and vertical respectively. So one need image scaling in wide sense (scaling with rotation). The image is marked by colored points for scaling. Zero point is scaled by red color. Top and right image points are marked by green and blue color respectively. Detection of these points is performed by following steps. First of all image is filtered. Following component represents filter or red color
RedColorFormula.png
In this formula rgb are weights of red, green and blue color respectively. Colors are normed by following way;

  • |r|≤1;
  • |g|≤1;
  • |b|≤1.

Above formula makes black point on place of red point. Other pixels are white. Filtration result is contained in Zero Image component. This component is connected to Zero Selection component which transforms image to chart. Properties of this chart are presented below:
FilteredZeroSelection.png
This selection enables us to detect coordinates of red point. Nonlinear regression component Zero Regression is used for this purpose (Zero Regression is connected to Zero Selection). Similraly using filters and regression components one can define coordinates of top and left point. Top Processor and Right Processor are being used for this purpose. Usage of filter which defines black pixels enable us define following chart.

After these actions one can define math dependency of chart. Regression formula is presented below:
ChartRegressionFormula.png
Parameters abcdk should be estimated. Main Processor is nonlinear regression component which performs estimation. Following chart contains calculated regression result (red curve) and a set of approximated points (blue color).
ChartAndRegression.png
Testing of this scenario should be ordered by following way:
ChartReconstructionOrder.png
The Order column means order of test:

3.2.2 Test of sounds

Any virtual reality software should support sounds. Also it should support testing of sounds. Let us consider a sample of sound testing. This sample contains following tasks:

  • Determination of orbits of artificial satellites;
  • Calculation of motion parameters;
  • Blowing of words “up” and “down” at virtual intersection of equator. “Up” is phonated if satellite is moved from South hemisphere to North one and “down” is phonated otherwise.

Following picture represents components of this sample
SoundTestFull.png
Left part of above picture contains determination of orbit, right one is related to sounds. Following picture represents right part in details. This picture has following meaning

Condition parameter Meaning Sound file
Sound formula.Formula_1 = true Satellite intersects equator plane from South to North up.wav
Sound formula.Formula_2 = true Satellite intersects equator plane from North to South down.wav

Testing of sounds is performed by following way. Preparation of test imply saving of dictionary of sounds. Dictionary keys are points of sound times, dictionary values are names of sound files. Implementation of this test is presented below:

 /// <summary>
/// Test of sound
/// </summary>
[Serializable()]
class SoundTest : ISerializable, ITest
{
    #region Fields

    string name;
    double start;
    double step;
    int stepCount;
    /// <summary>
    /// Dictionary of sounds
    /// Keys are instants of sounds
    /// Values are sound filenames
    /// </summary>
    Dictionary<double, string> soundResults = new Dictionary<double, string>();
    #endregion

    #region Ctor

    /// <summary>
    /// Constructor
    /// </summary>
    /// <param name="collection">Collection of components</param>
    /// <param name="name">Sound component name</param>
    /// <param name="start">Start time</param>
    /// <param name="step">Step</param>
    /// <param name="stepCount">Count of steps</param>
    internal SoundTest(IComponentCollection collection,
        string name, double start, double step, int stepCount)
    {
        this.name = name;
        this.start = start;
        this.step = step;
        this.stepCount = stepCount;
        soundResults = GetSounds(collection);
    }
    /// <summary>
    /// Deserialization constructor
    /// </summary>
    /// <param name="info">Serialization info</param>
    /// <param name="context">Streaming context</param>
    private SoundTest(SerializationInfo info, StreamingContext context)
    {
        name = info.GetString("Name");
        start = info.GetDouble("Start");
        step = info.GetDouble("Step");
        stepCount = info.GetInt32("StepCount");
        soundResults = info.GetValue("Sounds", typeof(Dictionary<double, string>)) as Dictionary<double, string>;
    }
    #endregion

    #region ISerializable Members

    /// <summary>
    /// ISerializable interface implementation
    /// </summary>
    /// <param name="info">Serialization info</param>
    /// <param name="context">Streaming context</param>
    void ISerializable.GetObjectData(SerializationInfo info, StreamingContext context)
    {
        info.AddValue("Name", name);
        info.AddValue("Start", start);
        info.AddValue("Step", step);
        info.AddValue("StepCount", stepCount);
        // Dictionary of sounds saving
        info.AddValue("Sounds", soundResults, typeof(Dictionary<double, string>));
    }
    #endregion

    #region ITest Members

    /// <summary>
    /// Testing
    /// </summary>
    /// <param name="collection">Collection of objects</param>
    /// <returns>Tesing result</returns>
    object ITest.this[IComponentCollection collection]
    {
        get
        {
            List<string> l = new List<string>();
            Dictionary<double, string> d = GetSounds(collection); // Calculates dictionary of sounds
            // Comparation of saved dictionary with calculated one
            if (d.Count != soundResults.Count)
            {
                l.Add("Different number of sounds");
            }
            foreach (double key in d.Keys)
            {
                if (!soundResults.ContainsKey(key))
                {
                    l.Add("Illegal time: " + key);
                }
                else
                {
                    string s = d[key];
                    if (!s.Equals(soundResults[key]))
                    {
                        l.Add("Illegal sound '" + s + "' Time = " + key);
                    }
                }
            }
             if (l.Count == 0)
            {
                return null;
            }
            // Returns list of errors
            return l;
        }
   }
    #endregion

    #region Members

    /// <summary>
    /// Name
    /// </summary>
    internal string Name
    {
        get
        {
            return name;
        }
    }
    /// <summary>
    /// Calculates dictionary of sounds
    /// </summary>
    /// <param name="collection">Collection of objects</param>
    /// <returns>Dictionary of sounds</returns>
    Dictionary<double, string> GetSounds(IComponentCollection collection)
    {
        IDesktop desktop = collection as IDesktop;
        Dictionary<double, string> d = new Dictionary<double, string>();
        SoundCollection coll = desktop.GetObject(name) as SoundCollection;
        IDataConsumer dc = coll;
        ITimeMeasureProvider p = DataPerformer.StaticExtension.Factory.TimeProvider;
        Action<string> act = (string s) => { d[p.Time] = s; };
        coll.PlaySound += act;
        dc.PerformFixed(start, step, stepCount, p,
           DifferentialEquationProcessor.Processor, desktop, 0, () => { });
        coll.PlaySound -= act;
        return d;
    }
    #endregion
}

If application is started with test support then test component have additional tab page of test’s parameters.
SoundTestTabPage.png
This page contains start time, count of steps and step of sound test. Test result depends on operation order. Dictionary of sounds depends on determination of orbit. If we would like save above scenario then we obtain following request of test.
SoundTestSaving.png
This request has following meaning

  • First of all test performs orbit determination with 3 iterations;
  • Secondly test of sounds is performed.

This sample requires installation of helper files. Following archive should be unpacked to directory of Aviation.Wpf3D.Advanced.exe file.

4. Points of interests

I find that testing is like to mammillae of man. Testing is not useful and it do not have beauty. However without advanced software is impossible without testing . Testing is inevitable misfortune.

5. Acknowlegment

I would like to acknowledge Weifen Luo for his very useful software. His software is used in my projects.

License

“Simple paradigm of scientific software testing”의 117개의 댓글

  1. I was recommended this web site via my cousin. I’m now not positive whether this
    submit is written by him as nobody else know such particular about my difficulty.
    You are wonderful! Thanks!

  2. I’m excited to discover this website. I want to to thank you for your time for this fantastic read!!
    I definitely liked every little bit of it and I have you book marked
    to see new information on your site.

  3. An impressive share! I’ve just forwarded this onto a coworker who had been doing a little
    homework on this. And he in fact bought me lunch due
    to the fact that I found it for him… lol. So allow me to reword
    this…. Thank YOU for the meal!! But yeah, thanks for
    spending time to talk about this topic here on your internet site.

  4. I just want to say I am just beginner to blogs and actually loved you’re web site. More than likely I’m planning to bookmark your blog . You really come with fabulous articles and reviews. With thanks for sharing your web site.

  5. Area on with this article, I absolutely think this web site needs much more factor to consider. I?ll probably be once again to review much more, many thanks for that details.

  6. Hmm it seems like your site ate my first comment (it was extremely long) so I guess I’ll just sum it up what I wrote and say, I’m thoroughly enjoying your blog. I too am an aspiring blog blogger but I’m still new to everything. Do you have any tips and hints for novice blog writers? I’d definitely appreciate it.

  7. Howdy would you mind stating which blog platform you’re using? I’m going to start my own blog soon but I’m having a hard time choosing between BlogEngine/Wordpress/B2evolution and Drupal. The reason I ask is because your design and style seems different then most blogs and I’m looking for something completely unique. P.S My apologies for getting off-topic but I had to ask!

  8. I?m satisfied, I must state. Really rarely do I encounter a blog site that?s both informative and also enjoyable, and also let me tell you, you have actually struck the nail on the head. Your suggestion is exceptional; the concern is something that inadequate people are speaking smartly around. I am extremely happy that I stumbled across this in my look for something connecting to this.

  9. Aw, this was an actually good message. In idea I would love to place in composing such as this furthermore? requiring time and also actual effort to make a great article? yet what can I claim? I procrastinate alot and also by no means seem to obtain something done.

  10. There are certainly a great deal of details like that to consider. That is an excellent indicate bring up. I provide the thoughts above as general motivation yet plainly there are questions like the one you bring up where one of the most essential thing will certainly be operating in truthful good faith. I don?t recognize if finest techniques have arised around points like that, yet I make certain that your job is plainly determined as a fair game. Both young boys and also girls really feel the impact of simply a moment?s pleasure, for the remainder of their lives.

  11. Spot on with this review, I absolutely think this web site requires much more consideration. I?ll probably be once more to review a lot more, many thanks for that details.

  12. I?d need to consult you here. Which is not something I typically do! I appreciate reviewing a blog post that will certainly make people believe. Additionally, thanks for permitting me to comment!

  13. Good article. I find out something much more challenging on various blogs everyday. It will certainly constantly be stimulating to check out web content from various other authors and also exercise a little something from their shop. I?d prefer to use some with the material on my blog site whether you don?t mind. Natually I?ll offer you a web link on your internet blog site. Many thanks for sharing.

  14. Please let me know if you’re looking for a article author for your weblog.
    You have some really good posts and I believe I would be
    a good asset. If you ever want to take some of the load off, I’d absolutely love to write some material for
    your blog in exchange for a link back to mine. Please send
    me an e-mail if interested. Thanks!

  15. The next time I check out a blog, I hope that it does not disappoint me as much as this set. I indicate, I know it was my choice to read, however I actually thought youd have something intriguing to say. All I hear is a number of yawping about something that you might take care of if you werent too busy searching for focus.

  16. The next time I read a blog site, I really hope that it doesn’t disappoint me as long as this set. I suggest, I recognize it was my selection to review, however I actually thought youd have something intriguing to state. All I hear is a bunch of yawping regarding something that you could take care of if you werent too busy searching for focus.

  17. Heya i’m for the primary time here. I came across this board and I in finding It truly useful & it helped me out a lot. I’m hoping to offer something again and help others like you aided me.

  18. Youre so cool! I do not suppose Ive review anything similar to this before. So good to discover somebody with some initial ideas on this subject. realy thanks for beginning this up. this internet site is something that is needed on the web, somebody with a little creativity. useful work for bringing something new to the web!

  19. When I originally commented I clicked the -Inform me when brand-new comments are added- checkbox as well as now each time a comment is added I obtain 4 emails with the exact same remark. Exists any way you can eliminate me from that service? Thanks!

  20. I and also my guys appeared to be looking at the excellent key points found on your web site and so suddenly I had an awful feeling I never expressed respect to the blog owner for those tips. All the young men ended up certainly thrilled to study all of them and have in effect seriously been taking pleasure in them. Many thanks for genuinely well kind and for figuring out this sort of quality subject areas millions of individuals are really wanting to know about. My very own sincere regret for not saying thanks to you sooner.

  21. A lot of thanks for every one of your effort on this web site. My niece delights in setting aside time for investigations and it is easy to understand why. My partner and i know all relating to the dynamic form you create very useful tricks by means of your web blog and as well as welcome response from people on that subject plus our simple princess is truly studying a whole lot. Have fun with the remaining portion of the year. You have been carrying out a really good job.

  22. Thanks for your own hard work on this website. My mother delights in managing investigation and it is easy to understand why. Almost all learn all regarding the lively mode you make great tips via the blog and as well improve participation from some other people on this concern and our princess is in fact becoming educated a lot of things. Have fun with the remaining portion of the year. You’re the one doing a brilliant job.

  23. I happen to be commenting to make you understand of the useful encounter my cousin’s child obtained checking your blog. She mastered many issues, with the inclusion of what it is like to have an incredible coaching spirit to make many others quite simply know specified impossible things. You undoubtedly exceeded our expectations. Thanks for displaying such good, dependable, informative as well as easy thoughts on this topic to Gloria.

  24. I and my friends were found to be taking note of the nice points from your web page and so quickly came up with an awful suspicion I had not expressed respect to the web blog owner for those secrets. All of the men happened to be totally stimulated to read through all of them and have now absolutely been tapping into those things. Thanks for getting simply kind and also for obtaining these kinds of superb subject matter millions of individuals are really needing to know about. Our own sincere regret for not expressing appreciation to sooner.

  25. I wish to express my appreciation for your kindness in support of men and women who actually need help on this particular theme. Your real commitment to passing the solution all around was surprisingly insightful and has enabled most people like me to realize their endeavors. Your amazing useful help implies much to me and extremely more to my office colleagues. Thank you; from each one of us.

  26. I wish to show my love for your generosity supporting those who require assistance with your idea. Your very own commitment to passing the solution across had become remarkably interesting and has regularly permitted professionals like me to arrive at their goals. Your amazing insightful suggestions implies this much a person like me and much more to my office colleagues. Thank you; from each one of us.

  27. I’m also commenting to make you be aware of of the great experience my wife’s princess encountered studying yuor web blog. She came to understand such a lot of issues, which included what it is like to have a marvelous teaching heart to let other individuals very easily fully understand a variety of multifaceted matters. You undoubtedly surpassed my desires. I appreciate you for coming up with those good, healthy, edifying and as well as easy tips about this topic to Mary.

  28. Thank you for all your valuable efforts on this site. My daughter takes pleasure in engaging in investigations and it’s easy to see why. Many of us hear all relating to the dynamic means you offer both useful and interesting ideas via this website and therefore welcome participation from others about this point plus our girl is understanding a great deal. Enjoy the rest of the year. You are always carrying out a brilliant job.

  29. I together with my pals were reading through the best strategies from your site and quickly I got an awful feeling I never expressed respect to the website owner for those tips. These women were as a result passionate to learn them and now have quite simply been having fun with these things. We appreciate you genuinely indeed thoughtful and for finding this sort of brilliant ideas millions of individuals are really wanting to learn about. Our own sincere regret for not saying thanks to you sooner.

  30. I really wanted to make a quick word so as to say thanks to you for the wonderful concepts you are showing here. My prolonged internet investigation has at the end been recognized with excellent know-how to write about with my guests. I would mention that many of us website visitors are really lucky to live in a fine place with so many perfect people with great secrets. I feel quite grateful to have seen your entire website page and look forward to many more awesome times reading here. Thanks a lot once again for all the details.

  31. Thanks so much for giving everyone a very breathtaking opportunity to check tips from this blog. It can be so useful and stuffed with fun for me and my office peers to search your website particularly thrice every week to find out the latest tips you have. And indeed, we are actually fulfilled with your staggering hints you give. Certain two ideas in this post are essentially the finest we have all ever had.

  32. My husband and i were really delighted that Louis managed to conclude his survey through your precious recommendations he grabbed when using the blog. It is now and again perplexing to simply find yourself releasing concepts which often other people may have been making money from. We really see we’ve got the writer to give thanks to because of that. The main illustrations you made, the straightforward site menu, the friendships you help to instill – it is many great, and it’s really making our son and us imagine that this subject matter is awesome, and that’s pretty fundamental. Thank you for everything!

  33. My husband and i felt absolutely peaceful that Ervin managed to complete his survey from the ideas he obtained while using the web page. It’s not at all simplistic just to continually be giving out tactics which some other people may have been trying to sell. And we all see we have got the writer to be grateful to for this. The type of illustrations you made, the straightforward web site menu, the relationships you make it easier to foster – it’s got all exceptional, and it is assisting our son in addition to us reason why the idea is cool, and that’s unbelievably pressing. Thank you for the whole lot!

  34. I am writing to make you know of the superb encounter our daughter experienced reading through yuor web blog. She learned too many issues, which include what it is like to possess a great teaching spirit to have other people really easily gain knowledge of several advanced subject matter. You truly did more than visitors’ expectations. Thanks for offering the beneficial, dependable, explanatory and also cool tips about your topic to Tanya.

  35. Thanks a lot for giving everyone such a spectacular chance to read from this web site. It’s usually very useful and packed with amusement for me and my office fellow workers to visit your blog particularly three times a week to see the latest guides you have got. And of course, I’m so usually astounded with your cool tactics you give. Certain two areas in this posting are easily the very best I’ve had.

  36. Great post. I find out something more challenging on different blog sites day-to-day. It will certainly constantly be stimulating to read web content from other authors and practice a little something from their store. I?d prefer to make use of some with the web content on my blog site whether you don?t mind. Natually I?ll offer you a web link on your internet blog site. Many thanks for sharing.

  37. I am just commenting to make you understand of the exceptional experience my cousin’s princess found checking your webblog. She figured out a good number of issues, with the inclusion of how it is like to have a very effective helping nature to have most people really easily know chosen tricky subject matter. You really did more than my expectations. Thanks for delivering those invaluable, trusted, educational and as well as unique tips about your topic to Lizeth.

  38. I simply desired to thank you very much once more. I’m not certain the things that I could possibly have tried without the entire smart ideas documented by you concerning such a theme. It truly was the horrifying problem in my opinion, nevertheless seeing a new well-written way you treated it forced me to cry with delight. Now i’m grateful for your service and in addition believe you are aware of an amazing job your are doing teaching the others all through a blog. Probably you haven’t encountered all of us.

  39. Thanks a lot for providing individuals with an exceptionally wonderful chance to check tips from this web site. It’s usually very sweet plus jam-packed with amusement for me and my office mates to search the blog at minimum 3 times weekly to see the fresh tips you have. And definitely, I’m so certainly astounded with your surprising techniques you serve. Certain 1 areas on this page are particularly the most efficient I’ve ever had.

  40. I want to express my passion for your kind-heartedness for individuals who really want help with this one content. Your real dedication to passing the message all around appeared to be amazingly good and has specifically allowed folks like me to arrive at their ambitions. Your interesting advice can mean a whole lot to me and a whole lot more to my office colleagues. Best wishes; from all of us.

  41. I intended to post you one bit of word to say thanks a lot over again for all the breathtaking tips you’ve shared on this page. This has been simply incredibly generous of you to offer openly all a few individuals might have advertised for an e-book to get some profit for their own end, and in particular now that you might well have done it in the event you considered necessary. These good tips as well worked as a fantastic way to realize that many people have similar eagerness the same as my own to learn a good deal more when it comes to this condition. I’m certain there are many more pleasant instances up front for folks who looked over your site.

  42. Thank you for every one of your work on this blog. Gloria take interest in getting into investigations and it’s easy to understand why. Many of us learn all regarding the lively ways you give informative tips and tricks via your blog and as well as attract response from other individuals about this area while our child has always been starting to learn a great deal. Have fun with the remaining portion of the new year. You’re doing a splendid job.

  43. This internet site is actually a walk-through for every one of the info you wanted regarding this and didn?t recognize who to ask. Look right here, as well as you?ll most definitely discover it.

  44. I have to express my appreciation for your kind-heartedness for those people who require help with the topic. Your personal commitment to getting the message all over turned out to be exceedingly important and have without exception helped employees just like me to achieve their desired goals. Your new useful suggestions signifies a lot a person like me and far more to my mates. Thank you; from everyone of us.

  45. Thanks for all of the hard work on this site. My mother takes pleasure in conducting research and it is easy to see why. A number of us know all about the powerful method you offer rewarding tips via this blog and even encourage contribution from other people on this situation and our princess is actually becoming educated a whole lot. Have fun with the remaining portion of the year. Your conducting a really great job.

  46. I would like to show thanks to the writer just for rescuing me from such a circumstance. As a result of looking throughout the the web and coming across tricks which are not pleasant, I thought my entire life was well over. Being alive without the solutions to the problems you have solved as a result of your good website is a critical case, and the kind which could have adversely affected my entire career if I hadn’t discovered the website. Your own personal skills and kindness in dealing with all the things was valuable. I don’t know what I would have done if I had not discovered such a stuff like this. I am able to at this moment relish my future. Thank you so much for your impressive and sensible guide. I will not be reluctant to recommend the sites to any person who wants and needs care about this subject matter.

  47. I must convey my affection for your kind-heartedness in support of those people who should have guidance on this one content. Your personal dedication to passing the solution throughout appeared to be really interesting and has specifically made ladies like me to attain their endeavors. Your interesting tips and hints indicates so much a person like me and extremely more to my office colleagues. Thanks a lot; from everyone of us.

  48. I simply needed to appreciate you once more. I am not sure the things that I might have worked on without those thoughts discussed by you directly on that industry. It had become a real terrifying matter in my circumstances, nevertheless being able to see the specialized fashion you processed it forced me to cry with joy. Extremely happy for your advice and even trust you realize what a great job you are always providing instructing others thru your blog. I am sure you haven’t got to know all of us.

  49. I not to mention my pals were analyzing the nice information and facts found on your site then the sudden I had a horrible feeling I had not expressed respect to the website owner for those tips. These people came so excited to learn them and already have in fact been enjoying those things. Appreciate your really being simply helpful and for picking some really good areas most people are really desperate to be informed on. My honest regret for not expressing gratitude to sooner.

  50. I not to mention my friends were digesting the best thoughts on your site while before long I had a terrible suspicion I had not expressed respect to you for those secrets. The young men came certainly passionate to read all of them and have now truly been tapping into them. Many thanks for truly being indeed accommodating and for making a decision on this sort of superb useful guides most people are really desirous to learn about. My honest regret for not expressing appreciation to you sooner.

  51. I have to show some thanks to this writer for rescuing me from this challenge. Just after browsing throughout the online world and getting basics which are not productive, I thought my life was done. Existing without the presence of solutions to the problems you have resolved through your main guideline is a critical case, and those that might have adversely damaged my entire career if I hadn’t come across the website. Your own personal capability and kindness in taking care of all the details was invaluable. I’m not sure what I would have done if I hadn’t come upon such a solution like this. I can also at this time look forward to my future. Thank you so much for the high quality and sensible help. I won’t think twice to propose your web page to anybody who requires counselling about this topic.

  52. I simply needed to appreciate you once again. I do not know the things that I might have implemented without the entire tips provided by you over that topic. Entirely was a real depressing setting in my view, however , taking note of this expert form you solved that made me to cry over happiness. Extremely grateful for the help and then have high hopes you know what an amazing job your are putting in instructing people today via a blog. I am sure you’ve never got to know all of us.

  53. My husband and i were quite fortunate that Raymond could finish off his homework using the ideas he discovered through the weblog. It is now and again perplexing to just always be giving away thoughts which men and women might have been selling. And we acknowledge we’ve got you to appreciate for that. The explanations you made, the straightforward blog menu, the friendships your site make it possible to create – it’s got all spectacular, and it’s assisting our son in addition to the family understand this matter is cool, and that’s really indispensable. Thanks for the whole lot!

  54. I intended to compose you one very little remark to help thank you once again with the amazing pointers you’ve featured in this article. It’s strangely open-handed of you to make easily all that many of us could have distributed for an ebook to end up making some bucks for themselves, notably now that you might have done it in the event you considered necessary. Those strategies as well worked to become a great way to understand that most people have the same eagerness just like mine to figure out very much more on the topic of this matter. I am certain there are millions of more pleasurable opportunities ahead for folks who scan your site.

  55. I simply wanted to write a word in order to say thanks to you for some of the stunning solutions you are posting at this website. My time intensive internet look up has finally been compensated with wonderful points to write about with my colleagues. I would declare that many of us website visitors actually are rather blessed to be in a decent site with very many special people with interesting hints. I feel really fortunate to have discovered your weblog and look forward to some more exciting times reading here. Thanks once more for a lot of things.

  56. Thanks a lot for giving everyone remarkably spectacular opportunity to read from this web site. It’s always so ideal and also full of a good time for me and my office acquaintances to search your site at the least three times in one week to see the newest things you have. And of course, I’m so certainly motivated with the attractive creative ideas you serve. Selected 4 tips in this article are indeed the finest we have all had.

  57. I am only commenting to let you be aware of of the remarkable discovery my wife’s princess undergone viewing your site. She noticed so many pieces, not to mention how it is like to have a great teaching spirit to make other folks with ease have an understanding of specified very confusing matters. You actually did more than our own desires. Thanks for offering these precious, dependable, revealing and in addition easy tips about that topic to Lizeth.

  58. I am glad for writing to make you be aware of of the remarkable discovery my child enjoyed reading through your webblog. She picked up a wide variety of issues, including what it is like to have an incredible teaching character to make the mediocre ones quite simply fully understand certain complicated subject areas. You undoubtedly exceeded her desires. Many thanks for producing these warm and helpful, healthy, edifying and fun thoughts on your topic to Gloria.

  59. I needed to compose you that tiny word to finally say thanks a lot over again with your unique thoughts you have shared here. This has been simply surprisingly generous of people like you to make publicly what a few people might have marketed for an e-book to earn some cash for themselves, notably considering the fact that you could possibly have tried it if you desired. The guidelines additionally served as the great way to know that someone else have similar keenness just as my very own to realize more regarding this issue. I think there are numerous more fun sessions ahead for those who see your website.

  60. Thanks a lot for giving everyone such a pleasant possiblity to discover important secrets from here. It’s always very cool plus jam-packed with fun for me and my office co-workers to search your blog at minimum three times a week to learn the newest secrets you have got. And lastly, I’m just at all times satisfied considering the superb inspiring ideas you give. Selected 1 ideas in this post are basically the finest we’ve had.

  61. I must show appreciation to you for rescuing me from such a challenge. After scouting throughout the search engines and finding principles which are not powerful, I was thinking my life was done. Being alive without the approaches to the difficulties you have solved by way of your good posting is a crucial case, as well as ones that might have in a negative way affected my entire career if I had not encountered your website. Your personal expertise and kindness in maneuvering all the stuff was important. I don’t know what I would’ve done if I had not encountered such a solution like this. I’m able to at this point look ahead to my future. Thanks so much for your high quality and effective guide. I won’t hesitate to refer your web page to any individual who ought to have recommendations on this subject.

  62. I would like to voice my respect for your kindness giving support to those people who should have help on this particular issue. Your special commitment to getting the message throughout appears to be certainly beneficial and has frequently empowered somebody just like me to achieve their aims. Your personal informative suggestions means this much to me and a whole lot more to my office colleagues. Thanks a lot; from all of us.

  63. I have to point out my appreciation for your kindness supporting men and women that really need help with this one theme. Your special commitment to getting the solution along has been extremely beneficial and have truly empowered girls just like me to achieve their ambitions. Your amazing interesting report implies this much a person like me and substantially more to my office colleagues. Thanks a lot; from each one of us.

  64. I wanted to send a brief comment to be able to say thanks to you for all of the pleasant guides you are giving out on this site. My time-consuming internet investigation has at the end been honored with wonderful ideas to exchange with my guests. I would assert that many of us visitors actually are rather lucky to live in a decent site with very many special people with interesting principles. I feel truly lucky to have come across your webpage and look forward to so many more thrilling minutes reading here. Thanks again for a lot of things.

  65. I just wanted to write down a small remark to say thanks to you for all of the lovely advice you are showing on this site. My extensive internet investigation has finally been honored with excellent knowledge to exchange with my great friends. I would say that most of us website visitors are definitely fortunate to exist in a superb site with many special people with very beneficial concepts. I feel rather lucky to have seen the website page and look forward to so many more fun minutes reading here. Thank you again for all the details.

  66. I simply wished to appreciate you yet again. I am not sure what I might have carried out in the absence of the entire creative concepts provided by you regarding my industry. It truly was the alarming issue in my opinion, however , viewing a skilled approach you treated it took me to weep over gladness. Extremely happy for this guidance and expect you comprehend what a great job you have been getting into teaching most people via your blog. Most probably you have never come across all of us.

  67. I simply want to tell you that I’m very new to weblog and definitely loved this blog. Almost certainly I’m likely to bookmark your website . You actually come with tremendous articles. Thank you for sharing with us your web page.

  68. I just want to say I am just newbie to blogging and site-building and honestly enjoyed this website. Probably I’m planning to bookmark your website . You absolutely come with perfect posts. Bless you for sharing your webpage.

  69. You really make it seem so easy with your presentation but I find this matter to be actually something that I think I would never understand. It seems too complex and extremely broad for me. I’m looking forward for your next post, I’ll try to get the hang of it!

  70. I would like to thnkx for the efforts you’ve put in writing this web site. I am hoping the same high-grade site post from you in the upcoming also. In fact your creative writing skills has encouraged me to get my own web site now. Really the blogging is spreading its wings quickly. Your write up is a great example of it.

  71. I was wondering if you ever thought of changing the page layout of your site? Its very well written; I love what youve got to say. But maybe you could a little more in the way of content so people could connect with it better. Youve got an awful lot of text for only having one or 2 images. Maybe you could space it out better?

  72. hello there and thank you for your information – I have definitely picked up something new from right here. I did however expertise a few technical points using this web site, as I experienced to reload the site many times previous to I could get it to load correctly. I had been wondering if your web hosting is OK? Not that I am complaining, but slow loading instances times will often affect your placement in google and can damage your high-quality score if ads and marketing with Adwords. Anyway I am adding this RSS to my e-mail and could look out for much more of your respective interesting content. Ensure that you update this again soon..

  73. I’ve been browsing on-line greater than 3 hours nowadays, but I by no means discovered any fascinating article like yours. It is lovely worth sufficient for me. In my view, if all web owners and bloggers made good content material as you probably did, the net might be much more useful than ever before.

  74. I simply want to mention I am just newbie to blogging and actually enjoyed this web site. More than likely I’m going to bookmark your site . You surely have awesome articles. Many thanks for revealing your website page.

  75. You’re so interesting! I don’t think I’ve truly read through anything like that before. So nice to discover someone with original thoughts on this subject. Really.. many thanks for starting this up. This web site is one thing that’s needed on the web, someone with a bit of originality!

  76. Hello there! I know this is kind of off topic but I was wondering which blog platform are you using for this site? I’m getting sick and tired of WordPress because I’ve had issues with hackers and I’m looking at options for another platform. I would be great if you could point me in the direction of a good platform.

  77. Hey there! I could have sworn I’ve been to this site before but after browsing through some of the post I realized it’s new to me. Nonetheless, I’m definitely happy I found it and I’ll be book-marking and checking back frequently!

  78. The the very next time I read a weblog, Hopefully that this doesnt disappoint me as much as this one. I mean, I know it was my substitute for read, but I really thought youd have some thing fascinating to convey. All I hear is actually a handful of whining about something that you could fix should you werent too busy trying to find attention.

  79. I have to say i am very impressed with the way you efficiently website and your posts are so informationrmative. You have really have managed to catch the attention of many it seems, keep it up!

  80. I really love your blog.. Great colors & theme. Did you develop this site yourself? Please reply back as I’m hoping to create my own personal site and want to learn where you got this from or just what the theme is named. Cheers.

  81. I truly love your blog.. Great colors & theme. Did you build this site yourself? Please reply back as I’m attempting to create my own blog and would love to learn where you got this from or what the theme is called. Many thanks.

댓글 남기기