Step 1 : Normalization

2015-10-05 16:25:04   8  举报





Normalization是一种数据处理技术，主要用于将不同尺度或范围的数据转换为统一的标准。这个过程可以应用于各种数据类型，包括文本、图像和音频等。在文本处理中，Normalization通常涉及到将文本转换为小写、去除标点符号和其他非字母字符、词干提取等步骤，以便于后续的分析和处理。在图像处理中，Normalization可能包括将像素值缩放到0-1的范围、进行白化处理以消除光照变化的影响等。总的来说，Normalization是数据预处理的重要步骤，它可以帮助我们更好地理解和利用数据。

作者其他创作

大纲/内容

Street name Normalization

If Street name begins with 0

Try with Short Form if long format is given or long format whether short form is given

If street name contains FM/Farm Market and number

Remove the 0 that begins and only street name should present in the field.

Prefix of FM and Suffix of FM number should be removed. FM-xxxx will be exact format.

If street name contains text Highway or Hwy and number

All the special characters and values inside closed bracket needs to be removed

Prefix of Hwy and Suffix of Hwy number should be removed and highway/Hwy should be replaced by their short form of state. TX-35 will be the correct format.

If street name contains any special characters or values inside braces