1、tf.nn.embedding_lookup()

一般做自然语言相关的。需要把每个词都映射成向量，这个向量可以是word2vec预训练好的，也可以是在网络里训练的，在网络里需要先把词的id转换成对应的向量，这个函数就是做这件事的

在基于深度学习的实体识别中，字向量会提前训练好，这个就可以理解成上面的tensor，而在实际的句子中每一个字所对应的字向量是通过id进行关联上的

例子：


#coding:utf-8
 
import tensorflow as tf
 
import numpy as np
 
c = np.random.random([5,1])  ##随机生成一个5*1的数组
 
b = tf.nn.embedding_lookup(c, [1, 3]) ##查找数组中的序号为1和3的
 
with tf.Session() as sess:
 
    sess.run(tf.initialize_all_variables())
 
    print(sess.run(b))
 
    print(c)
输出的结果如下所示：

[[0.5687709 ]

 [0.61091257]]

 

[[0.31777381]

 [0.5687709 ]

 [0.1779548 ]

 [0.61091257]

 [0.65478204]]

在c中第2个元素为0.5687709，第4个元素是0.61091257(索引从0开始)，刚好是b的值

2、tf.contrib.crf

functions

crf_binary_score(...): Computes the binary scores of tag sequences.
crf_decode(...): Decode the highest scoring sequence of tags in TensorFlow.
crf_log_likelihood(...): Computes the log-likelihood of tag sequences in a CRF.
crf_log_norm(...): Computes the normalization for a CRF.
crf_multitag_sequence_score(...): Computes the unnormalized score of all tag sequences matching tag_bitmap.
crf_sequence_score(...): Computes the unnormalized score for a tag sequence.
crf_unary_score(...): Computes the unary scores of tag sequences.
viterbi_decode(...): Decode the highest scoring sequence of tags outside of TensorFlow.

训练过程

Tensorflow 中 tf.contrib.crf.crf_log_likelihood 用于计算crf_loss，
在 bi-lstm + crf 或 idcnn + crf 结构中作为crf的网络的损失函数。

import tensorflow as tf
from tensorflow.contrib.crf import viterbi_decode
from tensorflow.contrib.crf import crf_decode

score = [[
    [1, 2, 3],
    [2, 1, 3],
    [1, 3, 2],
    [3, 2, 1]
]]  # (batch_size, time_step, num_tabs)
transition = [
    [2, 1, 3],
    [1, 3, 2],
    [3, 2, 1]
]   # (num_tabs, num_tabs)
lengths = [len(score[0])]   # (batch_size, time_step)

# numpy
print("[numpy]")
np_op = viterbi_decode(
   score=np.array(score[0]),
   transition_params=np.array(transition))
print(np_op[0])
print(np_op[1])
print("=============")

# tensorflow
score_t         = tf.constant(score, dtype=tf.int64)
transition_t    = tf.constant(transition, dtype=tf.int64)
lengths_t       = tf.constant(lengths, dtype=tf.int64)
tf_op = crf_decode(
    potentials=score_t,
    transition_params=transition_t,
    sequence_length=lengths_t)
with tf.Session() as sess:
    paths_tf, scores_tf = sess.run(tf_op)
    print("[tensorflow]")
    print(paths_tf)
    print(scores_tf)

[numpy]
[2, 0, 2, 0]
19
=============
[tensorflow]
[[2 0 2 0]]
[19]

tf.contrib.crf.crf_log_likelihood(inputs, tag_indices, sequence_lengths, transition_params=None)
函数的目的：使用crf 来计算损失，里面用到的优化方法是：最大似然估计，即在一个条件随机场里计算标签序列的log_likelihood

参数说明

inputs: [batch_size, max_seq_len, num_tags] ,一般使用BiLSTM处理之后输出转化为它要求的形状作为crf层的输入；
tag_indices: [batch_size, max_seq_len] 真实标签
sequence_lengths: [batch_size] 表示每个序列的长度
transition_params: [num_tags, num_tags]转移矩阵
# 返回值
log_likelihood: 标量，log_likelihood
transition_params:[num_tags, num_tags]转移矩阵

2、tf.einsum

1、job lib

import joblib
# 读取训练好的词向量语料库[418130,50]
filename_embeddings = "data/vecs.lc.over100freq.txt"
wordvectors, representationsize, words, wordindices = joblib.load(filename_embeddings + ".pkl")
print(representationsize)#每个字的维度50
print(words) # 所有字的集合
print(wordindices) # 每个字的索引:...'endifeq': 418127, '˚13': 418128, 'jaあ': 418129}

2、train_id_docs

self.train_id_docs = parsers.readHeadFile(self.filename_train)
self.dev_id_docs = parsers.readHeadFile(self.filename_dev)
self.test_id_docs = parsers.readHeadFile(self.filename_test)

def getCharsFromDocuments(documents):
    chars = []
    for doc in documents:
        for tokens in doc.tokens:
            for char in tokens:
                # print (token)
                chars.append(char)
    chars = list(set(chars))
    chars.sort()
    return chars